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Executive Summary 



Abstract 

A study examined the impact of the Qosing the Achievement Gap (GAG schools) 
program on the academic achievement of students in 30 schools serving high percentages of 
African American and economically disadvantaged students in West Virginia, 2004- 2008. 
The program assigned experienced educators, GAG liaisons, to the schools to provide school 
improvement coaching. The study used (a) a quasi- experimental approach to examine the 
differences between 12 GAG and 12 non- GAG matching schools for fourth- grade 
reading/ language arts and mathematics achievement on the statewide summative 
assessment; (b) a comparison of gains in the percentage of students vdio achieved 
proficiency for the same subject areas, fix>m GAG schools and all other WV schools during 
those years; (c) focus groups conducted with GAG liaisons to collect data about their work 
and their perceptions about impacts beyond academic achievement; and (d) GAG school 
faculty responses to three research- validated survey instruments that measured the schools' 
capacity for improvement, adherence to continuous school improvement processes, and 
overall school culture. Analysis of achievement testing showed promise for the program, 
especially for African American students. Results of the focus groups and surveys provide 
information about vdiich school improvement efforts received the most attention, and 
identified areas of strength and needed improvement in GAG schools at the end of the 5-year 
program. 



Background 

This report provides the results of a retrospective multimethod research study to 
determine the impact of the Qosing the Achievement Gap Professional Development 
Demonstration Schools (GAG schools) on the academic achievement of their students. This 
5-year (2004-2008) demonstration project was intended to develop and implement 
strategies to improve the achievement of aU students in 30 schools that served hi^ 
percentages of underachieving economically disadvantaged and African American students 
in West Virginia. The program used a school improvement coaching approach, assigning 
experienced educators, called GAG liaisons, vdio received specialized training to work vrith 
the GAG schools. The GAG liaisons worked to a set of standards developed by the West 
Virginia Department of Education, vdiich included (1) bringing focus, (2) leading change, (3) 
developing accountabihty, (4) building capacity, (5) creating community, and (6) growing 
professionally. 

This study was conducted by the West Virginia Department of Education (WVDE) 
Office of Researdii to address the follovring five research questions: 



1 Researchers did not receive funding from the office responsible for the CAG program except for the purchase of 
research instruments (i.e., surveys). 
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1. Is there a statistically significant difference between the average reading/ language 
arts and mathematics scale scores for CAG schools Mfien compared rvith matched 
comparison schools? 

2. To vdiat extent have CAG schools closed the achievement gap for African American 
students, students rvith disabilities, and economically disadvantaged students Mfien 
compared vdth achievement of all students in the state of West Virginia? 

3. What has been the focus of the CAG haisons over the course of the project, and what 
do the haisons perceive the impact of the program to be? 

4. How do CAG schools compare with hi^ performing learning communities in terms 
of continuous school improvement practices? 

5. What are the areas of strength and concern in CAG schools in terms of school culture 
and capacity for improvement? 

Methodology 

The study included both quantitative and qualitative components as needed to 
investigate each of the five research questions: 

For Research Question 1 (RQl) WVDE researchers used a quasi- experimental 
approach vdth 12 CAG elementaiy schools serving as the experimental group and 12 
matched comparison schools serving as the control group (non-CAG schools). They 
examined the differences between the CAG and non-CAG schools for fourth grade 
reading/ language arts and mathematics achievement on the statewide summative 
assessment, WESTEST, for the years 2004 throu^ 2008. Analyses were conducted to 
examine the differences in cumulative achievement over all 5 years (i.e., repeated 
measures ANOVA) and for individual year-to-year pairings (i.e., independent t tests). 
Average scale scores for each school in years 2004 through 2008 were entered as the 
dependent variables in these analyses. Group membership- as either experimental (i.e., CAG 
schools) or control (i.e., non-CAG schools)- and time were the independent variables. 

For Research Question 2 (RQ2), WVDE researchers examined the extent to vdiich 
achievement gaps persisted in CAG schools from 2004 to 2008. Achievement gaps were 
defined as the difference between the percentage of aU West Virginia students in non-CAG 
schools ^o were at or above Mastery (proficiency) in reading/ language arts and 
mathematics and the corresponding percentage for the following subgroups within CAG 
schools: 1) aU students, 2) African American students, 3) economically disadvantaged 
students, and 4) students with disabihties. These analyses were conducted for each school 
and the entire state for all 5 years. 

For Researcii Question 3 (RQ3), focus groups were conducted with CAG liaisons to 
provide cjualitative data about the work they had completed over the course of the project 
and their perceptions of the impact of the program beyond academic aciiievement. These 
sernistructured interviews were conducted at the end of the program with all available CAG 
liaisons. WVDE researchers coded the responses into themes and identified key findings. 

For Research Question 4 (RQ4), a research-validated survey instrument developed 
by Edvantia, Inc., the Continuous School Improvement Questionnaire (CSIQ) was 
distributed to all faculty and staff in the CAG sciiools at the end of the school year in 20 10 to 
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measure their perceptions about the extent to v^ch each CAG school possessed features 
common to high-performing learning communities, that is, schools that have received 
national recognition as schools of excellence and have demonstrated a commitment to 
continuous improvement. 

For Research Question 5 (RQ5), two other research- vahdated instruments developed 
by Edvantia, Inc., the Perceptions of School Culture (POSC) and Measure of School Capacity 
for Improvement (MSQ) were also distributed to all faculty and staff in the CAG schools 
during the same time period to measure perceptions about school culture, and the capacity 
of each of the CAG schools to succeed in school improvement efforts. The results of these 
two surveys were examined in comparison to a norm- referenced sample of similar schools. 
CAG schools were provided with a median percentile as a summary statistic. When possible, 
responses from the 2010 administration of the survey were compared with data collected in 
the same schools hem a 2006 administration. 

Findings 

As a result of these investigations, WVDE researchers found the follovdng: 

• RQl fin di ngs. Summative WESTEST assessment scores revealed no statistically 
significant differences between experimental (CAG) and control (non-CAG) 
elementary schools across the duration of the program. However, all groups 
examined in CAG schools— that is, African American, economically disadvantaged, 
and students with disabilities, and all subgroups combined— exhibited hi^er mean 
scale scores at the conclusion of the program than at its inception in 2004. This 
finding held true for both reading/ language arts and mathematics. With respect to 
the Black student subgroup, there was an increase in mean scale scores at CAG 
schools over time, vdiich ultimately resulted in Black students at the CAG schools 
exceeding the performance of Black students in non-CAG comparison schools for 
both reading/ language arts and mathematics in 2008. This finding of CAG schools 
outpacing comparison schools did not hold for the students with disabilities or 
economically disadvantaged subgroups or for all subgroups combined. 

• RQ2 findings. WVDE researchers found that as a group, the CAG schools exhibited 
cumulative proficiency level increases that exceeded statewide increases for the 
African American, economically disadvantaged, and students with disabilities 
subgroups, and all subgroups combined. This finding held for both reading/ language 
arts and mathematics. While the CAG schools exhibited more steady improvements 
in proficiency rates over the period of the program than the state as a whole, 
achievement gaps for the state were stiU narrower in 2008 than for CAG schools with 
respect to mathematics. Findings indicated that CAG sciiools were slightly more 
effective than the state in closing achievement gaps in mathematics for students vdth 
disabilities, but not for economically disadvantaged or Black students. 

• RQ3 findings. The liaisons reported that the majority of the tedinical assistance 
they provided had foc:used on addressing Standards 1, 4 and 6. Within these 
standards, five fod emerged, whicii induded (a) increased student learning and 
achievement, (b) providing guidance and support to schools as opposed to exphdt 
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direction, (c) increasing open communication rvith school staff, (d) facilitating and 
following up with professional development and (e) focusing on using data to drive 
instruction and assessment. 

The information from focus group interviews revealed the challenges and barriers 
associated with the GAG program including lack of preparation time during the day, 
requiring them to prepare for their work with teachers, principals, and county 
administrators in the evenings and on weekends. Other responses to the question 
about challenges or barriers to their work included the following: (a) getting in the 
door, (b) money, (c) lack of leadership or resistance to leadership, (d) lack of support, 
(e) retirements, and (f) maintaining a positive culture. 

• RQ4 findings. When examining the results of the CSIQ, some notable differences 
emerged among programmatic levels. CAG elementary and middle schools were 
generally comparable to or higher performing than the average elementary schools in 
terms of capacity for improvement. However, GAG high schools performed below the 
mean for average norm- referenced high schools on all seven subscales. The GSIQ 
identified at least three areas of strength for GAG schools as a group: (a) the presence 
of an ahgned and balanced curriculum, (b) a positive learning culture, and (c) hi^ 
quahty and purposeful student assessment. The GSIQ identified at least two critical 
areas of need for GAG schools as a group: (a) building school- farnily-cornmunity 
connections and (b) increasing school- wide capacity for effective teaching. 

• RQ5 findings. The POSG identified at least two areas of strength for GAG schools: 
the presence of a strong student- centered vision, mission, and policies and hi^ 
levels of teacher responsibility for learning. It also identified at least two potential 
areas of need for the aggregated group of GAG schools: building a sense of student 
responsibility for learning and ensuring the presences of an inviting physical 
environment. 

The MSQ identified at least three areas of strength for GAG schools as a group: the 
use of differentiated instruction, presence of technical resources, and adoption of a 
coordinated curriculum. It also identified three critical areas of need for GAG schools 
as a group: building capacity for peer reviewed practice, increasing expectations for 
student performance, and fostering equity in practice. 

Recommendations 

The WVDE Office of Research makes the following broad recommendations given 
the findings presented in this report: 

WVDE researchers recommend continued monitoring of the achievement gaps in 
GAG schools and statewide. The Office of Research is poised to accomplish this work via the 
annual updating of the Qosing the Achievement Gap report. 

Despite a lack of statistical significance for the fourth- grade assessment data 
examined, the GAG program was associated with a variety of successful program outcomes 
including marginally hi^er post- test perceptions of school culture, capacity for 
improvement, and continuous school improvement on most item subscales. Additionally, for 
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the majority of cases, the GAG schools made steady progress in narrovring achievement gaps 
for all subgroups in both reading/ language arts and mathematics. While the research study 
conducted is not sufficient to determine whether the CAG program caused these changes, it 
is likely that the support provided by the CAG program was helpful for CAG schools. It is 
also evident that there is still room for improvement in these schools. The Office of Research 
recommends continued support in order to capitahze on this success. The CAG liaisons have 
already built substantial rapport with these schools and would be very valuable in these 
future efforts. 

If the CAG program continues, ongoing evaluation should be conducted from the 
outset in order to more rigorously determine the program's success in achieving its 
formative and summative goals. Because the data for this study were compiled 
retrospectively, it is impossible for WVDE researchers to determine a causal hnk between 
the CAG program and the outcomes reported herein. While another evaluation would likely 
be subject to similar deficiencies due to the lack of a true experiment, it would certainly 
allow for a more credible study of program impact. Additionally, ongoing evaluation is 
critical to guiding programmatic adjustments to the services delivered via the program. 

The current study revealed a great deal of valuable information about the areas of 
strength and areas of concern in individual CAG schools. This information could be 
potentially useful in guiding future intervention efforts in the CAG schools. The Office of 
Research has provided these results to the program director and recommends that they be 
shared with the individual schools involved in the CAG program and perhaps with other 
offices vrithin the WVDE that are working vrith these schools to impact school improvement. 
Specifically, the areas of strength identified by this study could be used as leverage points 
vrithin these schools and areas of concern could be used to provide a focus for additional 
support services. 

The information from focus group interviews revealed the challenges and barriers 
associated with the CAG program. WVDE should learn from these challenges and design 
intervention programs that are better suited to avoid these potential pitfalls. 

Limitations 

This study was retrospective in nature. The hmited availability of some data imposed 
constraints on the analyses WVDE researchers were able to conduct. Staff did not employ a 
methodology adequate to make causal inferences about the impact of the CAG program. This 
report is intended to stand among other evidence collected by the WVDE Office of Title II, 
III, and System Support during the course of the CAG program, and should be viewed as a 
single component rather than as the entire portfolio of evidence. 
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Introduction 



The achievement gap in education is usually described as a gap in academic 
peifonnance, between racial/ ethnic minority and disadvantaged students and their White peers 
(National Governors Association, 2003). Accorxling to Olson (2004), the achievement gap can 
be seen in students' grades, standardized test scores, course selection, hi^ school dropout rates, 
college completion rates, other academic areas, and in diminished job opportunities and hfetime 
earnings potential. The median lifetime income for someone between the ages of 24 and 64 who 
does not earn a hi^ school diploma is about $1 million. If that same person obtains his or her 
hi^ school diploma, the median income rises to $1.2 milhon, and if that person completes some 
junior college or college education and/or training after high school, then the median income 
rises to approximately $1.5 million (Olson, 2004). Achieving these education milestones 
depends in large measirre on the success of students in mastering the skills and gaining the 
knovdedge required to advance through their school years. 

The Center on Education Policy (CEP, 2009) recently released the third armual report in 
a multiyear study of student achievement. The report provides a thorough analysis of student 
statewide accountability assessment data disaggregated by subgroup (vdienever possible) from 
all 50 states. CEP examined grades 4, 8, and 10 state test results at three achievement/ 
performance levels: (a) basic- and- above, (b) proficient- and- above, and (c) advanced. For their 
analyses, CEP treated West Virginia's WESTEST performance level partial mastery as basic; 
WESTEST mastery as proficient; and WESTEST above mastery -I- distinguished as advanced. 
The researchers then determined the extent to vdiicir students from different racial/ ethnic and 
socioeconomic backgrounds had made gains within all three performance levels and/ or Mfrether 
progress was lagging at any performance level for specific subgroups. 

Overall, CEP found that West Virginia, like most of the country, had made progress in 
closing adhievement gaps for the various subgroups identified in the No Child Left Behind Acf 
(NCLB). In West Virginia during the 2004 to 2008 period the adhievement gap narrowed for 
African American (compared with White) dhildren and economically disadvantaged children 
(compared with their more advantaged peers) at grades 4 and 8 in mathematics. At grade 10, 
gains in mathematics acirievement for African Americans were similar to Whites (meaning there 
was no progress in dosing the gap), rdrile the achievement gap continued to narrow for 
economically disadvantaged students. There were similar results for reading, vdth African 
American and economically disadvantaged students narrowing the achievement gap in aU three 
grades, except for African American eighth graders, vdio lost a little ground (-0.3%) compared 
with their White peers' gains. Although these are generally encouraging results, the achievement 
gaps remain large, ranging from a 7- to 20-point difference in the percentage of students scoring 
at profident or above, as shown in Table 1. 
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Table 1. West Virginia Subgroup Achievement Trends in Reading and Mathematics by Percentages Proficient 
or Above 



Grade 4 Grades Grade 10 



Proficient Proficient Average Proficient Proficient Average Proficient Proficient Average 

or Above or Above Annual or Above or Above Annual or Above or Above Annual 



Subgroup 


2004 


2008 


Gain 


2004 


2008 


Gain 


2004 


2008 


Gain 


Reaiding Achievement 


White 


74% 


83% 


2.3 


81% 


81% 


0.0 


77% 


74% 


-0.8 


African 

American 


58% 


76% 


4.5 


73% 


72% 


-0.3 


61% 


63% 


0.5 


Not low- 
income 


90% 


90% 


-0.1 


89% 


88% 


-0.3 


85% 


81% 


-0.9 


Low 

income 


65% 


76% 


2.8 


72% 


73% 


0.3 


65% 


63% 


-0.5 


Mathematics Achievement 


White 


70% 


77% 


1.8 


70% 


73% 


0.8 


64% 


69% 


1.3 


African 

American 


54% 


67% 


3.3 


50% 


59% 


2.3 


44% 


49% 


1.3 


Not low- 
income 


81% 


86% 


1.3 


80% 


82% 


0.4 


73% 


76% 


0.7 


Low 

income 


61% 


68% 


1.8 


58% 


63% 


1.3 


49% 


57% 


2.0 



SOURCE: Table WV-12 and Table WV-13 in State Profiles and Worksheets for State Test Score Trends Through 
2007-08, Subgroup Achievement and Gap Trends— West Virginia, published by Center on Education Policy. 
Retrieved from http://www.cep-dc.org/index.cfm ?fuseaction=Page.viewPage&pageld=572 . 



The Closing the Achievement Gap (CAG) Program 

In 2003 , West Virginia Governor Bob Wise created the Governor's Minority Students 
Strategy Council to investigate achievement gap issues in the state and make recommendations. 
The council completed its investigation, identifying achievement gaps in counties with high 
percentages of low income and minority students, and proposed legislation to create 
professional development (PD) schools as a demonstration project that could provide a model 
for other schools and districts in the state facing achievement gap issues (Kusimo, Petty- Wilson, 
&Body, 2004). 

The West Virginia Legislature passed legislation, vdiich was signed into law in 2004 
(School Law Article 2E. Hi^ Quahty Education Programs. §18-2E-3g. Special demonstration 
professional development school project for improving academic achievement). In the bill 
language, the Legislature made the following statements: 

(1) Well-educated children and families are essential for maintaining safe and 
economically sound communities; 
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(2) Low student achievement is associated with increased delinquent behavior, 
higher drug use and pregnancy rates, and higher unemployment and adult incarceration 
rates; 

(3) Each year, more students enter school with circumstances in their lives that 
sciiools are ill- prepared to accommodate; 

(4) Ensuring access for all students to the rigorous curriculum they deserve 
recjuires effective teaching strategies that indude, but are not hmited to, using a variety 
of instructional approaches, using varied curriculum materials, engaging parent and 
community involvement and support in the educational process, and providing the 
professional development, support and leadership necessary for an effective school; and 

(5) The achievement of aU students can be dramatically improved vdien schools 
focus on factors within their control, sudi as the instructional day, curriculum and 
teaching practices. 

The Population to Be Served and Schools Selected 

Among the provisions in the law was the establishment of a spedal 5-year demonstration 
project targeting elementary and middle schools located in 10 counties vdiere the enrollment of 
African American and/ or economically disadvantaged students was at least 5% (Table 2) . 

Table 2. County Ranking by Percentage of Black Subgroup (Highest Percentage to Lowest) and Percentage of 
Economically Disadvantaged Subgroup, 2008-2009 

Meidian 

Total householid 



County 


enrollment 

count* 


Black 

enrollment* 


Low SES 
enrollment* 


% Black 


% Low SES 


income, 

2008** 


West Virginia 


281,894 


15,200 


140,748 


5.4 


49.9 


$37,528 


Kanawha 


28,465 


3,718 


14,671 


13.1 


51.5 


$41,488 


Berkeley 


17,214 


2,091 


7,261 


12.1 


42.2 


$52,472 


McDowell 


3,675 


419 


2,903 


11.4 


79.0 


$20,486 


Raleigh 


12,316 


1,281 


6,325 


10.4 


51.4 


$34,589 


Jefferson 


8,398 


827 


2,579 


9.8 


30.7 


$62,802 


Mercer 


9,538 


924 


5,487 


9.7 


57.5 


$31,283 


Ohio 


5,279 


494 


2,421 


9.4 


45.9 


$38,344 


Cabell 


12,522 


1,116 


6,344 


8.9 


50.7 


$33,360 


Fayette 


6,810 


439 


3,806 


6.4 


55.9 


$32,934 


Marion 


8,122 


505 


3,917 


6.2 


48.2 


$35,630 



*West Virginia Department of Eiducation (2008). Counties by Composition, All Grades, All Subgroup - Sorted by 
County, School Year: 2008-2009. Retrieved from http://wveis.kl2.wv.us/nclb/pub/enroll/e06Makeup_county. 
cfmPso CNSN&grade=99&size0&updown=l&sop=0&spcd=T&sy=09 

**U.S. Census Bureau (n.d.) State & County Quickfacts. Washington , DC: Author Retrieved from http://quickfacts. 
census. gov/qfd/states/54000.html 
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The law required that at least three pubhc schools from each of the 10 counties become 
demonstration schools, for a total of 30 schools (Gilchrist, Trevisan, Salgado, and Holloway, 
2006). The selected schools would operate using an approach to professional development 
meant to address the particular needs of underachieving African American, economically 
disadvantaged, and disabled students, enabhng them to catch up with their White, economically 
better-off, and nondisabled peers. Passage of this legislation resulted in the development of the 
Closing the Achievement Gap Professional Development Demonstration Schools (GAG schools). 

Within the counties selected for the GAG schools project, the percentage of needy 
students ranged from a relative low of about 31% to a hi^ of 79%, with the median at 50% 
(Table 2). And ^Abile West Virginia statewide has a relatively small percentage of Black students 
(approximately 5.4% of the total student population or 15,200 students in 2008-2009), the 10 
counties represented in this study have the hipest percentages of Black pubhc school students 
(Table 2). 

Not all of the original 30 schools participated for the hill 5 years of the project due to 
various factors, including school consohdations, school sphtting (in an area of the state 
experiencing rapid grovdli), and the decision of one district (Ohio County) to discontinue 
participation after the first 2 years. By the end of the project, there were 26 participating 
schools. 

Program Funding 

The program was funded throu^ a special appropriation from the West Virginia 
Legislature, rAbich paid for school improvement specialists, called GAG haisons, and provided 
grants to counties that participated in the project. The grants to counties were for $20,000 a 
year, vriiich was split among the three schools to be served, and was used to pay for additional 
professional development and other spedahzed support needed in the schools. The GAG 
liaisons, vdio worked halftime, were paid by the state separately. After the first year or two, some 
counties opted to provide additional funding sufficient to support their CAG haison fulltime. 

Program Design 

Beginning in the mid 1990s, various states have experimented with an approach to 
providing professional development and other assistance to strug^ng schools by hiring and 
training experienced, highly successful educators to serve as school improvement specialists (or 
school improvement coaches). This intervention usually begins vrith the school improvement 
specialist helping the principal(s) and teachers in a low-performing school study data related to 
their own school (e.g, graduation rates of various subgroups, attendance rates, achievement 
tests scores, and the like) and identifying areas of needed improvement. With help from the 
school improvement specialist, the teachers and administrators then bring focus to their efforts 
by putting together an improvement plan that identifies a manageable number of issues to 
address in the coming year. Activities then focus on gaining the knovdedge, skills, and resources 
most needed to address the identified needs. Key to the success of these efforts is reserving time 
in the schedule for school faculties to meet and collaborate on a regular and ongoing basis, and 
estabhshing a professional learning community, vdiich vdll be involved in continuous 
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improvement by using data to drive their planning teaching, and improvement initiatives going 
forward. 

Kentucky and Tennessee have supported two of the most influential school improvement 
coaching programs. The earliest one is the Kentucky Distinguished Educator program (now 
known as the Highly Skilled Educator program), which began in the 1994-1995 school year. 
According to a report produced by the Southern Regional Education Board, schools were 
identified for this intervention because they had experienced serious declines in their 
performance. Their success after working with a distinguished educator was notable; of "the 53 
schools initially selected, 100% reversed their downward trend after two years of participation in 
[the program] and 63% exhibited student growth in excess of the expected rate of growth 
established for their school" (Mandel, 2000, p 3). Subsequent years also showed impressive 
success in turning around troubled schools, allowing schools to meet their NCLB adequate 
yearly progress goals. According to a 2006 report, "schools that received assistance through a 
combination of [highly skilled educator. Commonwealth School Improvement Fund] grant, and 
a scholastic audit or review showed statistically significant improvements in their accountability 
index scores" (Program Review and Investigations Committee, 2006, p. ix). The Commonwealth 
of Kentucky continues to support the program. 

Tennessee's Exemplaiy Educators program, established in 2000, draws upon the 
experience of retired teachers and administrators, hired throu^ a rigorous selection process. 
On average, exemplary educators have 34 years of experience and most have at least a master's 
degree. The skills and expertise of the exemplary educators are then further augmented through 
ongoing and intensive professional development (Edvantia, 2005). According to a recent report, 
42 schools served by exemplary educators were rolled off the state's high priority schools list in 
2008-2009, with 38 of them classified as being in good standing; and since 2001, 176 schools 
served by exemplary educators have achieved adequate yearly progress. The program, operated 
by a nonprofit research and development organization, Edvantia, was honored by Harvard 
University with a Top 50 Innovations in American Government award in 2007 (Bumgardner, 
2009). The Teimessee Exemplary Educator program, like the Kentucky Highly Skilled Educator 
program, continues to be supported by its sponsoring state education agency. 

Brief reports of similar programs have appeared in the literature, including programs in 
Alabama, Georgia, Louisiana, North Carolina, and South Carolina (Program Review and 
Investigations Committee, 2006) and in Maine (Kostin & Haeger, 2006). There is very little 
rigorous research in the literature, however, about the efficacy of any of these programs, 
although a series of descriptive and quasi- experimental evaluation studies of the Teimessee 
program has been conducted by Edvantia (see Craig, Butler, & Moats, 2005; Craig, J ., et al., 
2004a; 2004b; 2005a; & 2005b). 

The school improvement coaching program in West Virginia, known as the Closing the 
Achievement Gap Professional Development Schools program, was initially developed by the 
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WVDE Office of Title II, HI and System Support in collaboration with Edvantia. A description of 
how the program worked in West Virginia follows. ^ 

Professional development for teachers and administrators at CAG schools 

Professional development for teachers and administrators from the 30 CAG schools took 
place primarily in two settings: (a) statewide Leadership Team Conferences held twice a year, 
and (b) onsite in their own schools and districts. Principals (or their designees), teacher leaders, 
and CAG liaisons attended the Leadership Team Conferences, -vdiere they learned about such 
topics as the continuous improvement process, leading change, processes for teams to use to 
reach consensus ^en making decisions, data analysis, agenda building, keeping minutes, 
monitoring implementation of improvement plans, and so forth. Principals arrived a half day 
before the teacher leaders for these 3- day meetings, so they could be introduced to the purposes 
of the training and their roles in conducting it. The WVDE planned and conducted the 
conferences, vdiich featured training by staff firom the department and by nationally known 
providers hum such organizations as ASCD, Mid- Continent Research for Education and 
Learning (McREL), Solution Tree, and others. 

While the Leadership Team Conferences provided information about hi^ priority 
topics, helping teachers and administrators transfer that knovriedge to school practice was a 
large part of the professional development facilitated by CAG liaisons in the CAG schools. CAG 
liaisons observed dassrooms, demonstrated teaching techniques in dassrooms, partidpated in 
leadership team meetings and hdped teacher leaders and prindpals identify and prioritize other 
spedalized technical assistance and professional devdopment needed to address the needs of 
the individual schools they worked in. Additionally, they helped fadhtate the development of 
collaborative teams (also knovm as professional learning communities), organized by grade level 
or content area, and helped schools find vrays to allocate time in the school schedule for these 
teams to do their work. 

Selection and training of CAG liaisons 

The WVDE held an open recruitment for CAG haisons in 2004. Selection was based on 
rigorous criteria, induding demonstrated understanding and success in improving curriculum 
and instruction, increasing student learning, and leading change. WVDE sdected people who 
had been successful in improving their own schools, either as prindpals or district central office 
staff. Some were retired; others were hired from their administrative positions. 

The CAG liaisons were initially trained by Edvantia; trainers induded staff vdio had been 
instrumental in developing the Exemplary Educator program in Tennessee, induding one 
individual vriio had worked for several years as an Exemplary Educator. CAG liaisons received at 
least 10 days of training a year conduded by Edvantia or the WVDE. In later years of the 
program, CAG liaisons individually received spedahzed training based on the needs of the 
schools they served, which developed considerable expertise among the CAG haison cadre. 



2 Except viiiere noted, the description of the CAG program is based on an interview with Richard Lawrence, executive 
director of the WVDE Office of Title II, III, and System Support, on December 13, 2010. 
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All training for GAGS was grounded in the principles of the WVDE's Framework for Hi^ 
Performing 21 ^ Century School Systems, t^feich includes three components: (a) a cohesive and 
shared set of core beliefs that enhance staff performance on behalf of students; (b) the use of 
hi^-yield practices related to curriculum, instruction, school culture and organizational 
practices, and student and parent support; and (c) a continuous improvement process that 
utihzes data on an ongoing basis for setting priorities and strategic planning (WVDE, 2006) . 

GAG haisons were provided with a set of six standards under which they operated as they 
served their respective schools. The standards were largely based on the successful Tennessee 
Exemplary Educator program with adaptations made in collaboration with the WVDE staff to 
bring them into alignment with the Framework for Hi^ Performing 21 ^ Gentury School 
Systems. Edvantia and WVDE staff developed a professional development curriculum to support 
the operationalization of the six standards, vdiich served as the guiding framework for the 
project. Each standard had under it broad areas of assistance that the GAGs could provide to the 
schools they served, as follows: 

Standard 1: Bringing Focus 

• Professional development and assistance in writing and reviewing Strategic 5- Year 
Plans 

• Development of leadership teams in schools 

• Assistance with using data as the basis for school and dassroom dedsion making 

Standard 2: Leading Ghange 

• Greation of professional learning communities 

• Gdllaboration with instructional coaches/ demonstration teachers to model and teach 
21st century learning and instructional strategies 

• Greation of cooperative planning protocols to maintain focus on student learning 

Standard 3: Developing Accountabihty 

• Assistance with the use of formative and benchmarking assessments for student 
learning 

• Fadlitating cooperative planning as a vehide for improving instruction 

• Working with administrators and teachers on using data for program and instrud- 
tional evaluation 

Standard 4: Building Gapadty 

• Work with staff to understand and implement standards-based curriculum and in- 
structtion that is driven by data 

• Professional development for prlndpals and staff on 21st century teaching and 
learning 

• Fadhtating the habit of reflective practice by school staff 

• Supporting conditions in which schools can dose the achievement gap 

Standard 5: Greating Gommunity 

• Fadhtating cooperative planning 
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• Book studies on culture and community 

• Working cooperatively with local colleges/ universities and the West Virginia 
Achievement Project 

Standard 6: Growing Professionally 

• Participation in year-long professional development as a foundation for school 
improvement 

• Engaging in on- going reading and research on topics appropriate to our schools 

• Professional development on 21st century learning (West Virginia Department of 
Education, n.d., p. 3) 

Level of involvement ofCAG liaisons 

The WVDE did not prescribe a seguence or priority to the standards, instead allowing 
CAG liaisons to decide ^at approach to take in focusing their efforts based on the needs of the 
individual schools they served. In most cases, CAG liaisons worked with three schools located 
within each participating county. They generally worked one hill day in each of their assigned 
schools, each week. Their travel was paid for, and they worked out of their homes vdien they 
were not in schools. 



Purpose of the Study 

This report provides the results of a retrospective multimethod research study to 
determine the impact of the CAG schools program on the academic achievement of students and 
to describe implementation issues encountered by the CAG haisons. The study, vdiich included 
both quantitative and qualitative components, was conducted by the West Virginia Department 
of Education (WVDE) Office of Research. ^ Five broad research questions guided the study: 

1. Is there a statistically significant difference between the average reading/ language 
arts and mathematics scale scores for CAG schools M^en compared with matched 
comparison schools? 

2. To what extent have CAG schools closed the achievement gap for Black, special 
education and economically disadvantaged students rA^en compared with 
achievement of aU students in the state of West \firginia? 

3. What has been the focus of the CAG liaisons over the course of the project and what 
do the liaisons perceive the impact of the program to be? 

4. How do CAG schools compare with high performing learning communities in terms 
of continuous school improvement practices? 

5. What are the areas of strength and concern in CAG schools in terms of school culture 
and capacity for improvement? 



3 Researchers did not receive funding from the office re^onsible for the CAG program except for the purchase of 
research instruments (i.e., surveys). 
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The five questions guiding this study required differing research methods and focused on 
various participants in the Qosing the Achievement Gap (GAG) schools, including students, 
teachers, administrative staff, and GAG haisons. For this reason, the methodological approach 
taken for each question is described in turn. 

Research Question 1 

Is there a statistically significant difference between the average reading/ 

language arts and mathematics scale scores for GAG schools when compared 

with matched comparison schools? 

To address Research Question 1 (RQl) the WVDE researchers employed a quasi- 
experimental design with experimental and control groups. 

Participant characteristics 

The 12 schools in this part of the study had high percentages of economically 
disadvantaged students— in all but one case exceeding the state average of 50%— and relatively 
high percentages of African American students. AH schools were at the elementary level. 
Subgroups of interest were economically disadvantaged and African American students, and 
students with disabihties. 

Sampling procedures 

Selection of experimental school sample began with the full pool of 26 GAG schools. The 
research team then eliminated schools lacking a full record of longitudinal student data for all 
relevant subgroups to be examined, reducing the pool of eligible schools to 20: 12 elementary 
schools, 7 middle schools, and 1 high school. Further considerations in selecting the sample 
included the fact that analyses of scale score data must be conducted within grade levels due to 
the presence of unique scales within each grade level, and the need for a minimum sample size 
of 10 schools that had enrollments of all study subgroups, in order for analyses to yield valid 
results. The research team selected the only programmatic level— elementary school^or Mfeich 
there were enou^ schools for analysis. As a result the final experimental sample consisted of 12 
GAG elementary schools. 

The 12 schools in the control group were selected based on the following stratified 
criteria to ensure that the control group closely matched the experimental group on these key 
variables: 1) percentage of students at poverty level, 2) location of schools, 3) grade span 
represented within the schools; 4) enrollment within each school; 5) percentage of students 
identified as having disabilities, and 6) percentage of students identified as African American. 

Measures and covariates 

Once all schools (N =24) had been identified, fourth-grade data fiom the WESTEST 
reading/ language arts and mathematics assessments were collected and analyzed for 12 GAG 
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schools (experimental) and 12 matched (control group) schools. The fourth grade was selected 
to ensure consistent interpretation of scores across schools. WESTEST data for school years 
2003-2004, 2004-2005, 2005-2006, 2006-2007, and 2007-2008 were analyzed with scale 
scores used as the unit of measure for analyses in all schools. Mathematics and reading/ 
language arts scale scores served as dependent variables; independent variables included 
condition (experimental or control), race/ ethnicity (African American), socioeconomic status 
(economically disadvantaged), and identification for special education services. 

Research design 

RQl sou^t to determine vdiether having CAG liaisons assigned to work with specific 
schools resulted in increased student performance on the statewide summative assessment over 
a 5-year period (2004-2008). Two different statistical tests were administered: (a) repeated 
measures analyses of variance (ANOVAs), and (b) t tests. Repeated measures ANOVAs were 
used to test the statistical significance of net changes in student achievement between the 
experimental and control groups over all 5 years of the project. In a sense, this test addresses the 
cumulative impact of the program on student achievement, both between and vdthin groups. 
However, this test does not address the statistical significance of any changes that occurred 
between specific pairs of academic years (e.g., 2004 to 2005, 2005 to 2006, or 2005 to 2008). 
Additionally, the ANOVAs do not test for the significance of individual year-to-year changes. To 
satisfy these demands, a series of independent t tests were completed. 

Research Question 2 

To what extent have CAG schools closed the achievement gap for Black, special 

education and economically disadvantaged students when compared Avith 

achievement of all students in the state of West Virginia? 

Due to the limitations described above, which resulted in a sample consisting of 
elementary schools only, WVDE researchers chose to conduct a series of descriptive post hoc 
analyses that were inclusive of summative assessment data from all grade levels when 
addressing Research Question 2 (RQ2). 

Participant characteristics 

Students at all grade levels vdio attended CAG schools were the participants in this 

study. 

Sampling procedures 

The entire population of CAG schools is included in these analyses, with the comparison 
group being the entire population of West Virginia public school students. 

Measures and covariates 

WVDE researchers decided to use proficiency rates as a way to gain a valid school- wide 
statistic because, unlike scale scores, vdiich are scaled independently within each grade level and 
cannot be aggregated across grade levels, the percentage of students attaining proficiency can be 
reasonably aggregated across grade levels for purely descriptive purposes. Dependent variables 
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were proficiency rates in reading/ languages arts and mathematics testing; independent 
variables included condition (CAG schools cf. all non-CAG schools in West Virginia), 
racQ^ ethnicity (African American cf. all students), socioeconomic status (economically 
disadvantaged cf. nondisadvantaged), and identification for special education services compared 
to nonidentified students. 

Research design 

As a first step, the research team calculated the total number of students in each 
subgroup and grade levd at each of the GAG schools. WVDE researdiers next determined the 
number of students in each subgroup and grade level who had met or exceeded mastery on the 
reading/ language arts and mathematics subtests of the WESTESTT. These students were then 
aggregated across grade levels to yield a school- wide number of students in each subgroup that 
met or exceeded proficiency. This number was divided by the total number of students in each 
subgroup for vdiom assessment data were available. This yielded a school- wide percentage of 
students vv^o either met or exceeded proficiency in reading/ language arts and mathematics in 
2004, 2005, 2006, 2007, and 2008. This process was replicated for the entire state as a point of 
comparison. Proficiency rates for GAG schools were then plotted against proficiency rates for the 
entire state to determine the extent to vdiich GAG sciiools were able to dose the adiievement 
gap between key subgroups of students and the overall population of WV students. 

Research Question 3 

What has been the focus of the GAG liaisons over the course of the project and 

what do the liaisons percdve the impact of the program to be? 

Focus groups were conducted to address Researcii Question 3 (RQ3). 

Participant characteristics 

GAG liaisons, vdio were selected, trained, and worked in the GAG sdiools during the 5- 
year program, partidpated in this part of the study. GAG liaisons representing experiences at the 
dementary, middle, and hi^ school levels partidpated in focus groups. 

Sampiing procedures 

All GAG liaisons were recjuested to partidpate. 

Measures and covariate 

The WVDE researchers used a semistructured focus group interview protocol consisting 
of eight open-ended cjuestions (Appendix B) to guide the conversations. 

Research design 

On November 3, 2009, six WVDE researchers met at the West Virginia Gultural Genter 
in Gharleston, West Virginia, to lead focus group sessions with GAG liaisons during one of their 
regular meetings. Interviews occurred over the course of 2 hours. The WVDE researchers 
attended the meeting and interviewed small groups of 3 to 4 GAG liaisons. The purpose of this 
data collection session was to allow the WVDE researchers to learn more about vdiat the GAG 
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liaisons believed was the primary focus of their jobs, their roles and duties, vdiat they felt were 
the greatest accomplishments at the school! s) and districts they served, vdiat they saw as 
barriers to their work, and so forth. 

The data collection process began with a brief overview of the GAG Schools program by 
the program director and a self- introduction by each GAG liaison. Following the opening 
introductions, the GAGs were divided into groups of three to four liaisons and paired with two 
staff in each focus group. Two WVDE researchers were placed with each group of GAG liaisons 
so that one researcher could facilitate the conversation using the protocol questions, and one 
researcher could type notes from the discussions. Focus group sessions were then transcribed 
for analysis. In the analysis stage, WVDE researchers focused on identifying themes prevalent 
across all focus group sessions. A WVDE researcher read and reread the transcripts noting the 
repetition of specific words, phrases or ideas. These repeated words or phrases were coded, 
collected, and collapsed into relevant themes. Selected participant quotations supporting 
specific themes were preserved to provide an authentic voice. 

Research Question 4 

How do GAG schools compare with high performing learning communities in 
terms of continuous school improvement practices? 

Research Question 4 (RQ4) was addressed by conducting a survey using a validated, 
norm- referenced instrument, and interpreting the resulting descriptive statistics. 

Participant characteristics and sampling procedures 

All faculty and staff at all 26 GAG schools were asked to participate in the survey. 
Measures and covariates 

The Gontinuous School Improvement Questionnaire ([GSIQ] Meehan, Salgado, Gordey, 
Finch, Graig & Butler, 2006) was used to address RQ4. The GSIQ is a measure of schools' 
performance on seven dimensions critical to school improvement. The tool has demonstrated 
excellent reliability and validity through previous research (Meehan et al., 2006). The GSIQ 
contains 70 Likert- type response items vdiich comprise seven distinct subscales, described 
below: 

1. Learning Gulture "..reflects how well the culture of the school encourages 
learning by all— students, staff, and administrators." 

2. School- Family- Gommunity Gonnections "..reflects the degree to which a staff 
perceives that parents and community members are involved in and feel part of 
the school." 

3. Shared Leadership ". . reflects the extent to vdiich staff view leadership as being 
shared— 'vriiether school administrators dominate decision making or there are 
mechanisms for involving teachers, students, and parents." 

4. Shared Goals for Learning ". . assesses the extent to vdiich the school has dear, 
focused goals that are understood by all members of the school community." 
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5. Purposeful Student Assessment . reflects the extent to vdiich respondents view 
student assessment data as meaningful; use data to guide instructional decisions; 
and believe data are communicated to the greater school community, including 
teachers, parents, students, and the general community." 

6. Effective Teaching . measures the extent to which teacher practice aligns with 
research on effective teaching." 

7. Aligned and Balanced Curriculum "..reflects the extent to which professional 
staff members perceive the school's curriculum to be aligned and balanced." 
(Meehan et al., 2006, pp. 3- 4). 

Research design 

The survey was mailed to schools at the end of the 2009-2010 school year, and the 
forms were mailed back to WVDE. Data were compiled, analyzed, and then compared with 
normative data contained within the CSIQ User Manual and Technical Report (Meehan et al., 
2006). Specifically, for each of the seven subscales of the CSIQ, comparisons were made 
between the 26 CAG schools and data from a large normative sample of schools previously 
identified by Edvantia researchers as high performing learning communities (HPLCs) and all 
additional schools. HPLCs are defined as schools that "possess positive characteristics relative 
to both student performance and commitment to continuous learning and improvement on the 
part of professional staff of the school" (Meehan, et al., 2002, p. 26). The comparison of data 
from CAG schools with HPLCs was intended to illustrate vdiere the CAG schools stand when 
compared to a very rigorous benchmark. Additionally, CAG schools were compared to all 
additional schools identified in the CSIQ User Manual and Technical Report as a point of 
comparison between CAG schools and the average or typical school. It was expected that CAG 
schools would have mean scores that were generally comparable to or greater than at least 
average or typical schools, given the fact that the CAG schools have received considerable WVDE 
resources over the course of the program. 

Research Question 5 

What are the areas of strength and concern in CAG schools in terms of school 
culture and capacity for improvement? 

Two surveys were conducted to investigate Research (^estion 5 (RQ5), both surveys 
were conducted using validated and norm- referenced instruments. 

Participant characteristics and sampling procedure 

Faculty and staff at all 26 CAG schools were requested to participate in both surveys. 
School culture measures 

Measures and covariates 

To collect data related to school culture, the Perceptions of School Culture questionnaire 
([POSC] Covdey, Voelkel, Finch, & Meehan, 2006) was administered. The POSC is a tool to 
measure perceptions of six key components of school culture. The tool has demonstrated 
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reliability and validity throng previous research (Covdey, Voelkel, Finch, & Meehan, 2006). 
The POSC contains 62 Likert- type response items vdiich comprise six distinct subscales as 
described below: 

1. Collaborative Working Relationships . reflects the extent to vdiich faculty work 
together, trust and respect each other, have open channels of communication, 
and share leadership and responsibihty for problem solving and decision 
making." 

2. Student- Centered Vision, Mission, and Policies "..indicates the degree to which 
the school vision, mission, goals, and pohdes are dear and consistent with each 
other; incorporate high expectations for all students; and are communicated to 
staff students and parents." 

3. Student Responsibility for Learning "..measures faculty perceptions of their 
students' intrinsic motivation, persistence, awareness of their own learning 
strengths, and control over their own learning." 

4. Teacher Responsibihty for Learning ". . reflects the degree to vdiich faculty strive 
to improve teaching and learning, at both the individual and coUective levels, and 
share responsibihty for high levels of student learning." 

5. Inviting Physical Environment "..indicates the extent to vdiich the school's 
physical environment is perceived as dean, safe, and attractive." 

6. Students and Parents as Dedsion Makers "..assesses the degree to vdiich 
students and parents partidpate in planning and dedsion making that impact the 
school program." (Covdey, et al., pp. 2- 3). 

Research design 

The survey was mailed to schools at the end of the 2009-2010 school year, and the 
forms were mailed back to WVDE. Data fix)m the CAG schools were compiled, analyzed, and 
then compared with normative data induded in the POSC User Manual and Technical Report 
(Covdey, et al., 2006). Spedfically, for each of the six subscales of the POSC, comparisons were 
made between the 26 CAG schools and data from a large normative sample of schools previously 
compiled by Edvantia researchers. The comparison is intended to illustrate vdiere the CAG 
schools stand vdien compared to other schools from the same programmatic level. A percentile 
rank ranging from 1 to 99 was determined for each CAG school for each subscale of the POSC. 

WVDE researchers also compared the average POSC subscale scores for the 26 CAG 
schools to the mean subscale scores for all CAG schools as reported in the 2006 Edvantia 
research report, Basehne Study of Selected Professional Development Schools in West Virginia 
(Gilchrist, Trevisan, Salgado, & Holloway, 2006), which was based on an administration of the 
instrument at the beginning of the program. It should be noted that these comparisons were 
made solely for descriptive purposes. No conclusion can be made as to vdiether the results are 
statistically significant or directly attributable to the presence of the CAG program. 
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Capacity for school improvement measures 

Measures and covariates 

The Measure of School Capacity for Improvement ([MSQ] Hu^es, Copley, HotMey, & 
Meehan, 2006) is a measure of how a school's faculty perceives their school relative to seven 
areas that relate to continuous school improvement. The tool has demonstrated reliability and 
validity through previous research (Hughes, et al., 2006). The MSQ contains 58 Likert- type 
response items, vdiich comprise seven distinct subscales as described below: 

1. Equity in Practice "..assesses equitable practices in the school— specifically, 
responsive pedagogy and anti-discriminatoiy practices." 

2. Expectations for Student Performance "..items assess staff members' 
expectations of the students and their beliefs that all students can perform well 
academically." 

3. Differentiated Instruction ". . addresses instructional practices and strategies for 
reaching students of diverse learning needs." 

4. Improvement Program Coherence pertains to the "extent to vdiich improvement 
initiatives and efforts at a school are coordinated." 

5. Peer Reviewed Practice ". . explore! s] the observation and review by staff of their 
peers' work." 

6. Coordinated Curriculum ". . addresses the coordination of curriculum within and 
across grade levels at the school." 

7. Technical Resources concerns "instructional resources and materials, including 
vdiether staff possess or have immediate access to adequate materials and 
resources to achieve instructional objectives." (Hughes, et al., 2006, pp. 2- 3). 

Research design 

The survey was mailed to schools at the end of the 2009-2010 school year, and the 
forms were mailed back to WVDE. Data from GAG schools were compiled, analyzed, and then 
compared with normative data included in the MSQ User Manual and Technical Report 
(Hughes, et al., 2006). Specifically, for each of the seven subscales of the MSQ, comparisons 
were made between the 26 CAG schools and data from a large normative sample of schools 
previously identified by Edvantia researchers as average or typical schools. It was expected that 
CAG schools would have mean scores that were generally comparable to or greater than average 
or typical norm- referenced schools, given the fact that the CAG schools have received 
considerable WVDE resources over the course of the program. Furthermore, the mean scores for 
each of the seven subscales were compared to the mean scores obtained at the beginning of the 
CAG program in 2005. 
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Section I. Summative Assessment Results 



Research Question 1 

Is there a statistically significant difference between the average reading/ 
language arts and mathematics scale scores for CAG schools when compared 
with matched comparison schools? 

Two different statistical tests were administered: (a) repeated measures two-way 
analyses of variance (ANOVAs) to test the statistical significance of net changes in fourth- grade 
student achievement between the experimental and control groups, and (b) t tests to determine 
the statistical significance of any changes that occurred between specific pairs of academic years 
and to test for the significance of within- group changes. The tables, charts, and brief narratives 
provided on the following pages present the findings of these analyses. Results of the ANOVAs 
are presented first, followed by t test results. 

WESTEST mathematics ANOVA results 

All subgroups combined results 

Data were analyzed using a two-way ANOVA with repeated measures on one factor. 
Mauchly's test indicated that the assumption of sphericity had been met (x^ = 9.41, p >.40). The 
Time x Group interaction was not significant, F(2, 4) = 1.26, p = .439, = .054. The main effect 

for group was not significant either F(2, 1) = .621, p = .439, = .027. However, the main effect 

for time was significant, F(2, 4) = 12.098, p < .000, 77 ^ = .355 (Table 3). 



Table 3. All Subgroups Combined— Two-Way Analysis of Variance Results for Time, Group, and Time x Group 

Interaction Effects on Fourth-Grade WESTEST Mathematics Performance, 2004-2008 



Effect 


Type III sum 
of squares 


df 


Mean square 


F 


P 




Main (time) 


1846.01 


4 


461.50 


12.10 


0.00 


0.35 


Main (group) 


320.27 


1 


320.27 


0.62 


0.44 


0.03 


Interaction 














(time*group) 


192.88 


4 


48.22 


1.26 


0.29 


0.05 



Table 4 and Figure 1 indicate that some change occurred in mathematics mean scale 
scores for the experimental group (12 schools having CAG liaisons) from 2004 to 2008. The 
performance gap between experimental and control group schools narrowed to the point of 
having only a small gap (0.32 mean scale score points) remaining in 2008— down from a gap of 
4.75 mean scale score points in 2004. This performance gap narrowing suggests that the 
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presence of the GAG liaison within the experimental group schools over the 5-year period may 
have had a positive effect on all students' performance. This small change in performance may 
warrant further research. 

Table 4. All Subgroups Combined— Fourth-Grade WESTEST Mathematics Mean Scale Scores and Standard 

Deviations, 2004-2008 


Year 


Group 


N 

(Schools) 


Mean 
scale scores 


Standard 

deviation 


2004 


Experimental 


12 


636.73 


12.60 




Control 


12 


641.48 


12.10 


2005 


Experimental 


12 


642.09 


9.93 




Control 


12 


648.94 


12.21 


2006 


Experimental 


12 


643.53 


11.50 




Control 


12 


647.49 


11.56 


2007 


Experimental 


12 


648.92 


7.78 




Control 


12 


649.37 


16.53 


2008 


Experimental 


12 


650.20 


6.63 




Control 


12 


650.52 


11.82 




Data were analyzed using a two-way ANOVA with repeated measures on one factor. 
Mauchly’s test indicated that the assumption of sphericity had been violated 20.34, P 
<02), therefore degrees of freedom were corrected using Greenhouse- Geisser estimates of 
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sphericity (6 =0.73). The Time x Group interaction was not significant, F(2, 2.685) = .942, p 
= .418, rf = .04. The main effect for group was not significant either F(2, 1) = .335, p = .569, rf 
= .01. However, the main effect for time was significant, F(2, 2.685) = 8.448, p < .000, rf = .28 
(Table 5). 



Table 5. Black Subgroup— Two-Way Analysis of Variance Results for Time, Group, and Time x Group 

Interaction Effects on Fourth-Grade WESTEST Mathematics Performance, 2004-2008 



Effect 


Type III sum 
of squares 


df 


Mean square 


F 


P 




Main (time) 


3933.91 


2.68 


1465.11 


8.45 


.000 


0.28 


Main (group) 


194.26 


1.00 


194.26 


0.34 


.569 


0.01 


Interaction 














(time*group) 


438.78 


2.68 


163.42 


0.94 


.418 


0.04 



While there was no statistical significant difference between experimental schools and 
control schools for the Black subgroup. Table 6 and Figure 2 clearly indicate that some change 
did occur in the mathematics mean scale scores for the experimental group (10 schools having 
CAG liaison) fix>m 2006 to 2008. Between 2006 and 2007, the Black subgroups in experimental 
group schools not only performed on an equivalent par vdth the control group schools, but 
actually surpassed those schools' Black subgroup performance. This change in mean scale scores 
suggests that the presence of the CAG haison within the experimental schools over the 5-year 
period may have had a positive effect on Black students' performance. This change in student 
performance may warrant further research. 

Table 6. Black Subgroup— Fourth-Grade WESTEST Mathematics Mean Scale Scores and Standard Deviations, 

2004-2008 



Year 


Group 


N 

(schools)* 


Mean 
scale scores 


Standard 

deviation 


2004 


Experimental 


10 


627.94 


12.27 




Control 


10 


631.76 


14.42 


2005 


Experimental 


10 


634.63 


7.38 




Control 


10 


640.32 


17.35 


2006 


Experimental 


10 


634.25 


10.31 




Control 


10 


641.19 


12.97 


2007 


Experimental 


10 


643.78 


9.62 




Control 


10 


639.84 


19.47 


2008 


Experimental 


10 


649.40 


8.25 




Control 


10 


644.57 


17.26 



Note. *N=10 instead of 12 because two matched schools did not have 5 years of data for subgroup comparison. 
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Economically disadvantaged subgroup results 

Data were analyzed using a two-way ANOVA with repeated measures on one factor. 
Mauchly’s test indicated that the assumption of sphericity had been met (x^= 14.37, P >.H). The 
Time x Group interaction was not significant, F(2, 4) = .579, p = .679, = .026. The main effect 

for group was not significant either F(2, 1) = 1.139, p = .297, if = .049. However, the main effect 
for time was significant, F(2, 4) =3.669, p <.008, if =.143 (Table 7). 



Table 7. Economically Disadvantaged Subgroup— Two-Way Analysis of Variance Resuits for Time, Group, and 
Time x Group interaction Effects on Fourth-Grade WESTEST Mathematics Performance, 2004-2008 



Effect 


Type III sum 
of squares 


df 


Mean square 


F 


P 




Main (time) 


733.10 


4 


183.27 


3.67 


.008 


0.14 


Main (group) 


473.02 


1 


473.02 


1.14 


.297 


0.05 


Interaction 














(time*group) 


115.65 


4 


28.91 


0.58 


.679 


0.03 



While there is no statistical significant difference between experimental schools and 
control schools. Table 8 and Figure 3 do suggest that some change did occur in mathematics 
mean scale scores for the experimental group from 2006 to 2008 for the economically 
disadvantaged subgroup. Each year following 2006 testing, the economically disadvantaged 
student performance gap narrowed between experimental and control group schools. For 
example, in 2007, the gap between the two groups was 2.29 mean scale score points and in 



20 Examining the Effectiveness of Closing the Achievement Gap Professional Development 

Demonstration Schools 



Results 



2008, that number decreased to a gap size of 2.04 mean scale score points. This represented an 
achievement gap decrease of 0.25 mean score points between the economically disadvantaged 
students in that 1-year period for experimental and control groups. 



Figure 3. 



Economically Disadvantaged Subgroup— Change in Fourth-Grade WESTEST Mathematics Mean 
Scale Scores, 2004-2008 
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Table 8. 


Economically Disadvantaged Subgroup— 
and Standard Deviations, 2004-2008 


-Fourth-Grade WESTEST Mathematics Mean Scale Scores 


Year 


Group 


N 

(schools) 


Mean 
scale scores 


Standard 

deviation 


2004 


Experimental 


12 


636.41 


13.37 




Control 


12 


641.32 


12.96 


2005 


Experimental 


12 


638.17 


8.96 




Control 


12 


645.50 


9.90 


2006 


Experimental 


12 


639.34 


10.34 




Control 


12 


642.62 


10.53 


2007 


Experimental 


12 


643.86 


7.64 




Control 


12 


646.15 


15.67 


2008 


Experimental 


12 


644.37 


8.46 




Control 


12 


646.41 


10.58 



Overall, the performance gap between experimental and control group schools narrowed 
from 4.92 mean scale score points in 2004 to 2.04 mean scale score points in 2008; however, 
the control group schools continued to outperform the experimental group schools. This 
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peifonnance gap narrowing suggests that the presence of the GAG haison within the 
experimental group schools over the 5-year period may have had some positive effect on 
economically disadvantaged student performance. This change in performance may warrant 
further research. 

Students with disabilities subgroup results 

Data were analyzed using a two-way ANOVA with repeated measures on one factor. 
Mauchly’s test indicated that the assumption of sphericity had been met (x^= 13.45, P >-14). The 
Time x Group interaction was not significant, F(2, 4) = .223, p = .925, rf = .010; nor was the 
main effect for group, F(2, 1) = .008, p = .928, 77^ = .000. However, the main effect for time was 
significant, F(2, 4) =8.710, p < .000, 77^ = .284 (Table 9). 



Table 9. Students with Disabilities Subgroup— Two-Way Analysis of Variance Results for Time, Group, and 
Time x Group Interaction Effects on Fourth-Grade WESTEST Mathematics Performance, 2004-2008 



Effect 


Type III sum 
of squares 


d£ 


Mean square 


F 


P 




Main (time) 


7596.46 


4 


1899.12 


8.71 


.000 


0.28 


Main (group) 


11.90 


1 


11.90 


0.01 


.928 


0.00 


Interaction 














(time*group) 


194.37 


4 


48.59 


0.22 


.925 


0.01 



Table 10 and Figure 4 suggest some interesting comparisons between the experimental schools 
and the control schools; however, it is not dear what impact the GAG haison initiative mi^t 
have had on the performance of students with disabihties within the treatment schools. For 
example, beginning in testing year 2004, the experimental schools outperformed the control 
group schools by 2.73 mean scale score points. The experimental group schools continued on an 
upward performance tr^ectoiy for the next 4 years with only a shght increase in performance 
between 2007 and 2008. Overall, the experimental schools increased their mean scale score by 
19.87 points between 2004 and 2008. Despite this overall increase in performance, the 
achievement gap between experimental and control group schools widened to 3.14 mean scale 
score points in 2008, because the control group schools had increased their mean scale score by 
25.74 points between 2004 and 2008. Thus, the achievement gap between treatment and 
control schools widened despite the placement of a GAG haison in the treatment schools, 
suggesting that this initiative may not have had any appredable positive impact on students 
with disabilities within the treatment schools. 
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Table 10. 


Students with Disabilities Subgroup- 
Standard Deviations, 2004-2008 


—WESTEST Fourth-Grade Mathematics Mean Scale Scores and 


Year 


Group 


N 

(schools) 


Mean 
scale scores 


Standard 

deviation 


2004 


Experimental 


12 


613.26 


11.60 




Control 


12 


610.53 


37.82 


2005 


Experimental 


12 


622.29 


13.55 




Control 


12 


625.69 


18.62 


2006 


Experimental 


12 


627.62 


14.64 




Control 


12 


628.51 


16.86 


2007 


Experimental 


12 


632.70 


25.63 




Control 


12 


630.75 


26.60 


2008 


Experimental 


12 


633.13 


17.61 




Control 


12 


636.27 


17.59 



Figure 4. Students with Disabilities Subgroup— Change in Fourth-Grade WESTEST Mathematics Mean 

Scale Scores, 2004-2008 
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Note. Experimental group (N=12) and control group (N=12) 

WESTEST reading/language arts results 

All subgroups combined results. 

Data were analyzed using a two-way ANOVA with repeated measures on one factor. 
Mauchly's test indicated that the assumption of sphericity had been violated (x^= 23.39, P 
<006), therefore degrees of freedom were corrected using Greenhouse- Geisser estimates of 
sphericity (6= 0.60). The Time x Group interaction was not significant, F(2, 2.415) = .466, p 
= .666, = .021. The main effect for group was not significant either, F(2, 1) = .921, p = .348, 
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\f = .040 . However, the main effect for time was significant, F(2, 4) = 4.823, p<.001, =. 180 

(Table 11). 

Table 11. All Subgroups Combined— Two-Way Analysis of Variance Results for Time, Group, and Time x Group 
Interaction Effects on Fourth-Grade WESTEST Reading/Language Arts Performance, 2004-2008 



Effect 


Type III sum 
of squares 


df 


Mean square 


F 


P 




Main (time) 


1020.28 


4.00 


255.07 


4.82 


.001 


0.18 


Main (group) 


384.72 


1.00 


384.72 


0.92 


.348 


0.04 


Interaction 

(time*group) 


98.57 


2.42 


40.81 


0.47 


.666 


0.02 



While Table 12 and Figure 5 reveal that the experimental schools were never able to close 
the performance gap with the control group schools, it is worth noting that the gap did narrow 
between 2004 and 2008. In testing year 2004, the control group schools (n=12) outperformed 
the experimental schools (n=12) by 6.29 mean scale score points. In 2008, the control group 
schools outperformed the experimental schools by only 3.32 mean scale score points. This 
represented a decrease in the achievement gap of 2.97 mean scale score points. 

Over the 5 years (2004-2008) the experimental group schools increased thar 
performance by 10.03 mean scale points vdiile the control group increased their performance by 
only 7.06 mean scale points. These increases in student performance resulted in a narrowing of 
the performance gap; yet, the control group schools continued to outperform the experimental 
schools despite the placement of a CAG liaison within the treatment schools. This suggests that 
this initiative may have had only limited positive impact on all students within the treatment 
schools, i.e., experimental schools did increase mean scale scores between 2004-2008, but not 
to a degree that would dose the achievement gap between the two groups of schools 
(experimental and control). 

Table 12. All Subgroups Combined— Fourth-Grade WESTEST Reading/Language Arts Mean Scale Scores and 
Standard Deviations, 2004-2008 



Year 


Group 


N 

(schools) 


Mean 
scale scores 


Standard 

deviation 


2004 


Experimental 


12 


634.65 


16.47 




Control 


12 


640.94 


13.00 


2005 


Experimental 


12 


638.13 


8.85 




Control 


12 


642.92 


12.00 


2006 


Experimental 


12 


641.21 


9.47 




Control 


12 


642.32 


12.02 


2007 


Experimental 


12 


642.76 


8.72 




Control 


12 


645.16 


12.78 


2008 


Experimental 


12 


644.68 


6.96 




Control 


12 


648.00 


8.56 



24 Examining the Effectiveness of Closing the Achievement Gap Professional Development 

Demonstration Schools 



Results 




Black subgroup results 

Data were analyzed using a two-way ANOVA with repeated measures on one factor. 
Mauchly's test indicated that the assumption of sphericity had been violated 28.01, p 
<001), therefore degrees of freedom were corrected using Greenhouse- Geisser estimates of 
sphericity (6 =0.64). The Time x Group interaction was not significant, F(2, 2.571) = .252, p 
= .830, 772 = .011. The main effect for group was not significant either F(2, 1) = .013, p = .912, 77^ 
= .001. However, the main effect for time was significant, F(2, 2.571) = 5.591, p < .003, 77^ 
= .203 (Table 13). 



Table 13. Black Subgroup— Two-Way Analysis of Variance Results for Time, Group, and Time x Group 

Interaction Effects on Fourth-Grade WESTEST Reading/Language Arts Performance, 2004-2008 



Effect 


Type III sum 
of squares 


df 


Mean square 


F 


P 




Main (time) 


2372.88 


2.57 


922.95 


5.59 


.003 


0.20 


Main (group) 


3.64 


1.00 


3.64 


0.01 


.912 


0.00 


Interaction 


(time*group) 


107.01 


2.57 


41.62 


0.25 


.830 


0.01 



Table 14 and Figure 6 reveal that yriiile there were no statistically significant differences 
between the reading/ language arts performance of the experimental schools and the control 
group, there were some interesting findings. For example, in examining the Black student 
achievement gap between the two groups of schools (experimental and control), the 
performance gap actually increased over the 5 years 2004-2008. In 2004, the performance gap 
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was 2.19 mean scale score points with the control group outperforming the experimental 
schools. In 2008, the performance gap increased to 5.0 mean scale score points with the 
experimental schools outperforming the control group schools. 

Over the 5-year period (2004-2008), the experimental schools increased their 
performance by 13.96 mean scale score points with 7.71 of those points occurring between 2007 
and 2008. Meanvdiile, the control group schools increased their performance only 6.77 mean 
scale points over the 5 years (2004-2008). While both groups improved their academic 
performance over the 5 years, the experimental group schools had more than two times the 
increase in mean scale score points than the control group schools (7.71 points to 3.33 points 
respectively) between 2007 and 2008. 

While it is unclear vdiat prompted this bump in Black subgroup performance in the 
experimental schools between 2007 and 2008, it is possible that the addition of a CAG liaison 
into the treatment schools had some positive impact on the performance of Black students over 
the 5-year period. This change in performance may warrant further research. 

Table 14. Black Subgroup— Fourth-Grade WESTEST Reading/Language Arts Mean Scale Scores and Standard 
Deviations, 2004-2008 



Year 


Group 


N 

(schools) 


Mean 
scale scores 


Standard 

deviation 


2004 


Experimental 


10 


629.88 


5.52 




Control 


10 


632.07 


10.66 


2005 


Experimental 


10 


630.92 


8.34 




Control 


10 


629.91 


16.45 


2006 


Experimental 


10 


635.31 


6.68 




Control 


10 


638.58 


13.31 


2007 


Experimental 


10 


636.13 


11.66 




Control 


10 


635.51 


17.02 


2008 


Experimental 


10 


643.84 


5.64 




Control 


10 


638.84 


14.38 



Note. N=10 instead of 12 because two matched schools did not have 5 years of data for subgroup comparison. 
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Economically disadvantaged subgroup results 

Data were analyzed using a two-way ANOVA with repeated measures on one factor. 
Mauchly’s test indicated that the assumption of sphericity had been violated (x^ = 37.04, p 
<000), therefore degrees of freedom were corrected using Greenhouse- Geisser estimates of 
sphericity (6 = 0 . 51 ). The Time x Group interaction was not significant, F(2, 2.038) = .408, p 
= .671, = .018; neither were the main effect for group F(2, 1) = 2.22, p = .150, rf = .092, or 

the main effect for time, F(2, 2.038) = 1.583, p < .216, = .067 (Table 15). 



Table 15. Economically Disadvantaged Subgroup— Two-Way Analysis of Variance Resuits for Time, Group, and 
Time x Group interaction Effects on Fourth-Grade WESTEST Reading/Language Arts Performance, 
2004-2008 



Effect 


Type III sum 
of squares 


df 


Mean square 


F 


P 




Main (time) 


467.48 


2.04 


229.36 


1.58 


.216 


0.07 


Main (group) 


617.39 


1.00 


617.39 


2.22 


.150 


0.09 


Interaction 

(time*group) 


120.52 


2.04 


59.13 


0.41 


.671 


0.02 



While there is no statistically significant difference between experimental and control 
schools. Table 16 and Figure 7 do indicate that both groups did have a small increase in mean 
scale score points for the economically disadvantaged students over the 5-year period (2004- 
2008). The experimental group schools had a total increase of 5.39 mean scale score points 
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between 2004-2008 tAMIe the control group schools had a total increase of 1.97 mean score 
points between 2004-2008. 

Throughout the entire 5-year period (2004-2008), a performance gap existed between 
the two groups of schools for the economically disadvantaged subgroup with the control group 
always outperforming the experimental group. The performance gap did narrow from 7.07 mean 
scale score points in 2004 to 3.65 mean scale score points in 2008— a decrease in the 
achievement gap of 3.42 points. This performance gap narrowing suggests that the presence of 
the CAG liaison within the experimental group schools over the 5-year period may have had a 
small positive effect on student performance; however, this impact was not enough to allow the 
experimental schools to catch up with or surpass the control group schools' performance. This 
small increase in performance over 5 years may warrant further research. 

Table 16. Economically Disadvantaged Subgroup— Fourth-Grade WESTEST Reading/Language Arts Mean Scale 
Scores and Standard Deviations, 2004-2008 



Year 


Group 


N 

(schools) 


Mean 
scale scores 


Standard 

deviation 


2004 


Experimental 


12 


634.34 


17.57 




Control 


12 


641.41 


13.43 


2005 


Experimental 


12 


632.76 


6.71 




Control 


12 


638.53 


8.71 


2006 


Experimental 


12 


636.41 


6.93 




Control 


12 


637.61 


11.73 


2007 


Experimental 


12 


636.15 


9.56 




Control 


12 


641.13 


13.27 


2008 


Experimental 


12 


639.73 


6.33 




Control 


12 


643.38 


6.58 
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Figure 7. Economically Disadvantaged Subgroup— Change in Fourth-Grade WESTEST 
Reading/Language Arts Mean Scale Scores, 2004-2008 




2004 2005 2006 2007 2008 

Year 

Experimental Control 

Note. Experimental group (N=12) and control group (N=12) 



Students with disabilities subgroup results 

Data were analyzed using a two-way ANOVA with repeated measures on one factor. 
Mauchly’s test indicated that the assumption of sphericity had been met 13-22, p >.15). The 
Time x Group interaction was not significant, F(2, 4) = 1.417, p = .235, 77^ = .061. The main 
effect for group was not significant either F(2, 1) = .897, p = .354, 77^ = .039. However, the main 
effect for time was significant, F( 2, 4) =3.244, p <.016, 77^ =.129 (Table 17). 



Table 17. Students with Disabilities Subgroup— Two-Way Analysis of Variance Results for Time, Group, and 
Time x Group Interaction Effects on Fourth-Grade WESTEST Reading/Language Arts Performance, 
2004-2008 



Effect 


Type III sum 
of squares 


df 


Mean square 


F 


P 




Main (time) 


4745.464 


4 


1186.37 


3.24 


.016 


0.13 


Main (group) 


1509.338 


1 


1509.34 


0.90 


.354 


0.04 


Interaction 

(time*group) 


2072.887 


4 


518.22 


1.42 


.235 


0.06 



The presence of CAG liaisons within the experimental schools did not seem to ameliorate 
the problems that the students with disabilities subgroup had with academic performance in 
reading/ language arts. 



Table 18 and Figure 8 reveal that in 2004, the achievement gap between experimental 
schools and control group schools was 0.65 mean scale score points; however, by 2008, the 
performance gap had grovm to 12.02 mean scale score points. 
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Dtuing the 5 years (2004-2008), the control group schools increased thar mean scale 
score by 22.21 points. These 12 schools had a steady increase in performance over that time 
period with the largest point gain occurring from 2006-2008. On the other hand, the 
experimental group schools increased their mean scale score by only 10.84 points overall from 
2004 to 2008, and at times those 12 schools were erratic in their performance. For example, the 
experimental schools decreased 5.32 mean scale score points between testing year 2004 and 
testingyear 2005, increased 14.57 mean score points from 2005 to 2006, decreased 12.23 mean 
score points from 2006 to 2007, and finally increased 13.82 mean score points from 2007 to 
2008. There did not appear to be a positive trend in overall performance for the experimental 
schools. 



Table 18. Students with Disabilities Subgroup— Fourth-Grade WESTEST Reading/Language Arts Mean Scale 
Scores and Standard Deviations, 2004-2008 



Year 


Group 


N 

(schools) 


Mean 
scale scores 


Standard 

deviation 


2004 


Experimental 


12 


606.97 


13.62 




Control 


12 


607.62 


26.12 


2005 


Experimental 


12 


601.65 


11.54 




Control 


12 


610.57 


29.70 


2006 


Experimental 


12 


616.22 


17.59 




Control 


12 


611.40 


23.41 


2007 


Experimental 


12 


603.99 


44.74 




Control 


12 


622.68 


21.90 


2008 


Experimental 


12 


617.81 


22.93 




Control 


12 


629.83 


20.86 
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Summary of results of comparisons between experimental and control groups 

The reader is reminded that, vdiile every attempt was made to match experimental 
schools (those with a CAG liaison) with other similar schools for the control group, the research 
team was hmited by the small number of schools in West Virginia that have substantial 
enrollments of both African American and economically disadvantaged students. Consequently, 
the baseline (2004) mean scale scores were not as similar as might be expected if schools were 
more closely matched. In all but one case— mathematics scores for students with disabilities 
(control = 610.53, experimental = 613.26)— the control group scores in 2004 were higher than 
the experimental group scores by approximately 1 to 7 mean scale score points. In this context. 
Table 19 presents an at-a-^ance pictorial view of the overall findings from an analysis of mean 
scale scores. 

An examination of this table quiddy reveals that over the 5-year period 2004-2008, the 
Black subgroup was the only subgroup among the experimental schools to increase their 
academic performance— as reflected in fourth- grade WESTEST mathematics and 
reading/ language arts mean scale scores— to the point that they caught up to or surpassed the 
mean scale scores of the control group schools. In WESTEST mathematics, this increase was 
especially noticeable from testing year 2004 to 2005 (6.69 mean score points); 2006 to 2007 
(9.53 mean points); and 2007 to 2008 (5.62 mean points). From testing year 2005 to 2006, the 
Black subgroup increase was only 0.38 mean score points. In WESTEST reading/ language arts, 
the primary increase was from testing year 2007 to 2008 (8.33 mean score points) causing the 
Black subgroup in experimental schools to outperform the control group schools. 
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Table 19. Summary of Mean Scale Score Analyses for Fourth-Grade WESTEST Mathematics and 
Reading/Language Arts Performance, 2004-2008 



Subgroup 


Experimental 
schools increased 
mean scale scores 
2004-2008 


Experimental 
schools had larger 
increases in mean 
scale scores 
compared with 
control schools 
2004-2008 


Experimental 
schools had 
smaller increases 
in mean scale 
scores compared 
with control 
schools 2004-2008 


Experimental 
schools caught up 
to or surpassed 
control schools in 
mean scale scores 
2004-2008 


WESTEST Mathematics Performance 2004-2008 


All Subgroups Combined 


• 


• 






Black 


• 


• 




• 


Economically Disadvantaged 


• 


• 






Students with Disabilities 


• 




• 




WESTEST Reading/ Language Arts Performance 2004-2008 


All Subgroups Combined 


• 


• 






Black 


• 


• 




• 


Economically Disadvantaged 


• 


• 






Students with Disabilities 


• 




• 





Table 19 also reveals that, vdiile the students- with- disabihties subgroup in the 
experimental schools achieved increased mean scale scores for mathematics and 
reading/ language arts over the 5-year period, these increases were never enough to enable them 
to score at or above the level of the control group schools. 

Both the all subgroups combined and economically disadvantaged subgroup in the 
experimental schools experienced increased mean scale scores from 2004 to 2008 on both 
assessments; however, even though these experimental subgroups made greater gains in their 
increased mean scale scores, they did not match or surpass the control group scores. 

Statistical significance of year-to-year changes in mean scale scores 

In addition to conducting repeated measures, two-way ANOVAs that examined 
differences between the overall 5-year trend for experimental and control schools, WVDE 
researchers also conducted a series of independent t tests both between and within each of the 
groups. Between- groups analyses were conducted to determine if there were any significant 
differences between experimental and control schools for any sin^e academic year. For 
example, a between- groups t test was used to determine if the 2004 mean scale score for 
experimental schools was significantly different from the 2004 mean scale score for control 
schools. These analyses were completed for all 5 years for all four subgroups (i.e., 40 total tests). 
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Next, WVDE researchers conducted a series of within- groups t tests to determine 
vriiether any year-to-year differences in mean scale scores were significantly different for either 
of the two groups (i.e., experimental and control). For example, a within- groups t test was used 
to determine if the 2004 and 2005 mean scale scores for the experimental group were 
significantly different. These analyses were completed for all possible year-to-year combinations 
for both the experimental and control groups (i.e., 160 total tests). 

Table 20 presents the mean scale score for both experimental and control schools as well 
as the delta (i.e., difference) between the groups. Reading down each column illustrates the 
within- groups differences. Reading across the table rows illustrates the between- group 
differences. 



Table 20. Comparison of Within-Group and Between-Group Experimental and Control Mean Scale Scores for 
Fourth-Grade WESTEST Mathematics and Reading/Language Arts Assessment, by Subgroup, 2004- 
2008 





All mathematics 




Black mathematics 




Low-SES mathematics 




SWD mathematics 


Year 


Exp 


Cont 


Delta 


Year 


Exp 


Cont 


Delta 


Year 


Exp 


Cont 


Delta 


Year 


Exp 


Cont 


Delta 


2004 


636.73 


641.48 


-4.75 


2004 


627.94 


631.76 


-3.82 


2004 


636.40 


641.32 


-4.92 


2004 


613.26 


610.53 


2.73 


2005 


642.09 


648.94 


-6.85 


2005 


634.63 


640.32 


-5.69 


2005 


638.17 


645.5 


-7.33 


2005 


622.29 


629.69 


-7.40 


2006 


643.53 


647.49 


-3.96 


2006 


634.25 


641.19 


-6.94 


2006 


639.34 


642.62 


-3.28 


2006 


627.62 


628.91 


-1.29 


2007 


648.92 


649.37 


-0.45 


2007 


643.78 


639.85 


3.93 


2007 


643.46 


646.15 


-2.69 


2007 


632.70 


630.75 


1.95 


2008 


650.2 


650.52 


-0.32 


2008 


649.40 


644.57 


4.83 


2008 


644.37 


646.41 


-2.04 


2008 


633.13 


636.27 


-3.14 






All RLA 






Black RLA 






Low-SES RLA 




SWD RLA 




Year 


Exp 


Cont 


Delta 


Year 


Exp 


Cont 


Delta 


Year 


Exp 


Cont 


Delta 


Year 


Exp 


Cont 


Delta 


2004 


634.65 


640.94 


-6.29 


2004 


629.88 


632.07 


-2.19 


2004 


634.34 


641.41 


-7.07 


2004 


609.97 


607.62 


2.35 


2005 


638.13 


642.92 


-4.79 


2005 


630.92 


629.91 


1.01 


2005 


632.76 


638.53 


-5.77 


2005 


601.65 


610.57 


-8.92 


2006 


641.21 


642.32 


-1.11 


2006 


635.31 


638.58 


-3.27 


2006 


636.41 


637.61 


-1.20 


2006 


616.22 


611.40 


4.82 


2007 


642.76 


645.16 


-2.40 


2007 


636.13 


635.51 


0.62 


2007 


636.15 


641.13 


-4.98 


2007 


603.99 


622.68 


-18.69 


2008 


644.68 


648.00 


-3.32 


2008 


643.84 


638.84 


5.00 


2008 


639.73 


643.38 


-3.65 


2008 


617.81 


629.83 


-12.02 



Note. Exp = Experimental group (i.e., schools with a CAG liason); Cont = Control group schools; Low-SES = 
Economically disadvantaged; SWD = Students with disabilities; RLA = Reading/language arts. 



None of the 40 between- groups t tests returned significant results. That is to say, there 
were no statistically significant differences between the experimental and control groups when 
looking at each year in isolation. Appendix A contains a list of t test results for these analyses for 
each subgroup (see Tables A1-A40 vriiich present tests of HO 1 - HO 40) . 

Several of the within- groups t tests returned significant results. That is, there were 
several statistically significant differences vdien examining year-to-year differences within both 
the experimental and control groups. Table 21 presents the results of the t tests that were 
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statistically significant for WESTEST mathematics scores within the experimental group. ^ 
Notably, the differences between 2004 and 2008 for experimental schools were statistically 
significant for three of the four subgroups under examination. The only group for vriiich this 
difference was not statistically significant was the economically disadvantaged subgroup. 
However, this group did experience a statistically significant increase when comparing the 2005 
scale score to 2007. These data are not sufficient to make causal claims, but it is important to 
note the statistically significant and positive differences over time for these groups. 

Table 21. Statistically Significant Gains in Fourth-Grade WESTEST Mathematics Scores within the Experimental 
Group by Subgroup, 2004-2008 



Subgroup Years compared Scale score Gains Appendix/table reference 



All groups combined 


2004 


to 


2005 


5.36 


A41 


All groups combined 


2004 


to 


2006 


6.8 


A42 


All groups combined 


2004 


to 


2007 


12.19 


A43 


All groups combined 


2004 


to 


2008 


13.47 


A44 


All groups combined 


2005 


to 


2007 


6.83 


A46 


All groups combined 


2005 


to 


2008 


8.11 


A47 


All groups combined 


2006 


to 


2007 


5.39 


A48 


All groups combined 


2006 


to 


2008 


6.67 


A49 


Black 


2004 


to 


2005 


6.69 


A51 


Black 


2004 


to 


2006 


6.31 


A52 


Black 


2004 


to 


2007 


15.84 


A53 


Black 


2004 


to 


2008 


21.46 


A54 


Black 


2005 


to 


2007 


9.15 


A56 


Black 


2005 


to 


2008 


14.77 


A57 


Black 


2006 


to 


2007 


9.53 


A58 


Black 


2006 


to 


2008 


15.15 


A59 


Economically disadvantaged 


2005 


to 


2007 


5.69 


A66 


Students with disabilities 


2004 


to 


2005 


9.03 


Ml 


Students with disabilities 


2004 


to 


2006 


14.36 


Ml 


Students with disabilities 


2004 


to 


2007 


19.44 


A73 


Students with disabilities 


2004 


to 


2008 


19.87 


A74 


Note. Statistical significance determined using t tests. See referenced tables in the Appendix for full t test results. 

Table 22 presents the results of the t tests that were statistically significant for fourth- 
grade WESTEST reading/ language arts scale scores within the experimental group. These 



4 In the interest of brevity, the results for control schools have been resigned to Appendix A. 
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differences were statistically significant M^ien examining six year-to-year pairings. The 
differences between 2004 and 2008 were statistically significant and positive for all subgroups 
combined and the Black subgroup, but not for the students with disabilities or economically 
disadvantaged subgroups. However, the difference between 2005 and 2008 was significant for 
the economically disadvantaged subgroup. Notably, there were no year-to-year significant 
differences for the students with disabilities subgroup in experimental schools. 



Table 22. Statistically Significant Gains in Fourth-Grade WESTEST Reading/Language Arts Scores within the 
Experimental Group by Subgroup, 2004-2008 



Subgroup 


Years compared 


Scale score gains 


Appendix/table 

reference 


All groups combineid 


2004 to 2008 


10.03 


A84 


All groups combineid 


2005 to 2008 


6.55 


A87 


Black 


2004 to 2008 


13.95 


A94 


Black 


2005 to 2008 


12.92 


A97 


Black 


2006 to 2008 


8.52 


A99 


Low-SES 


2005 to 2008 


6.97 


A107 



Note. Statistical significance determined using t tests. See referenced tables in the Appendix for full t test results. 

Research Question 2 



To what extent have CAG schools closed the achievement gap for Black, special 
education and economically disadvantaged students when compared with 
achievement of all students in the state of West Virginia? 

As noted in the methodology section, WVDE researchers also conducted a series of post 
hoc descriptive analyses comparing the percentage of students in all grade levels achieving at 
Proficiency— which includes those students vriio scored at Mastery, Above Mastery, and 
Distinguished— at all 26 CAG schools with the statewide percentage. The purpose of these 
analyses was to provide additional information about the extent to which CAG schools narrowed 
the achievement gaps that existed between subgroups of interest and the population of all 
students in West Virginia. What follows are brief narrative explanations of these data, followed 
by graphical representations. 

Closing the achievement gap in reading/language arts 

Figure 9 presents the reading/ language arts adiievement trends among all WV students 
in non-CAG schools (i.e., the remainder of the state). Results are presented by all groups 
combined and by subgroup (Black, economically disadvantaged, and students with disabihties). 
These data serve as a point of comparison for students in GAG schools. The top trend line in 
Figure 9 indicates that, statewide proficiency levels increased slightly between 2004 and 2006 
(77.23% to 80.04%) but exhibited a plateau for years 2007 and 2008 (80.01% and 79.99%). 
Economically disadvantaged and Black students exhibited similar achievement gaps in 2004 
vriien compared to the state (-8.94% and - 11.81%, respectively). That achievement gap deca^ased 
to -7.21% for economically disadvantaged students and -6.53% for Black students in 2008. This 
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indicates that Black students' rate of improvement exceeded that of economically disadvantaged 
students across the state. Students with disabilities exhibited the greatest achievement gap 
(-43.48%) in 2004, which decreased to -38.20% in 2008. 



Figure 9. Trends in the Percentage of Students Scoring at Proficiency in WESTEST 

Reading/Language Arts in all West Virginia (Non-CAG) Schools, 2004-2008 




School Year 

> State {All Students) Proficiency Reading M State SES Proficiency Reading 

A State Black Proficiency Reading )( State SwD Proficiency Reading 

Note. SES = Economically disadvantaged; SwD = Students with disabilities 



Figure 10 depicts the comparison in reading/ language arts achievement trends among all 
students in the CAG schools, all students in non-CAG schools (i.e., the remainder of the state), 
and the subgroups of interest in CAG schools (Black, economically disadvantaged and students 
with disabilities). The top two trend lines in Figure 10 indicate that all students in CAG schools, 
on average, exhibited an achievement gap of approximately -3.37% in 2004, but closed that gap 
to -0.41% by 2008. Economically disadvantaged students and Black students exhibited an even 
greater achievement gap in 2004 when compared to the state (-12.06% and -16.02%, 
respectively). However, that achievement gap decreased to -7.24% for economically 
disadvantaged students and -5.35% for Black students in 2008, indicating that Black students' 
rate of improvement in CAG schools exceeded that of economically disadvantaged students 
and exceeded the rate of improvement of both Black and economically disadvantaged students 
in non-CAG schools. However, it is important to note that the Black and economically 
disadvantaged student populations overlap, making generalizations difficult. Students with 
disabilities exhibited the greatest achievement gap (-46.39%) in 2004, which decreased to 
-34.51% in 2008. This indicates that students with disabilities in CAG schools outperformed 
non-CAG students with disabilities with respect to proficiency score increases in 
reading/ language arts (please refer to Figure 9) . 
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Figure 10. Trends in the Percentage of Students Scoring at Proficiency in WESTEST 

Reading/Language Arts, in CAG Schools (Subgroups and All Groups Combined) 
Compared with All Groups Combined in All West Virginia (Non-CAG) Schools 
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Note. SES = Economically disadvantaged; SwD = Students with disabilities 



Figure 11 depicts the differences in reading/ language arts proficiency improvements 
between all groups combined in CAG and non- GAG schools, as well as differences in proficiency 
improvements between subgroups of interest (economically disadvantaged. Black, and students 
with disabilities) in GAG and non- GAG schools. 



Examining the Effectiveness of Qosing the Achievement Gap Professional Development 37 

Demonstration Schools 



Results 



Figure 11. Differences between WESTEST Reading/Language Arts Achievement Gains in CAG Schools 

(Subgroups and All Groups Combined) and All West Virginia (Non-CAG) Schools from 2004 to 
2008 

■ CAG Gain ■ State Gain 




All Groups Combined and Subgroups 
Note. SES = Economically disadvantaged; SwD = Students with disabilities 



Notably, the proficiency increase in CAG sdiools was higher than the state in all cases. 
Although the proficiency increase between aU students in CAG and non- CAG schools only 
differed by -f-2.95%, the proficiency increases among subgroups were higher. Economically 
disadvantaged students in CAG and non- CAG schools exhibited the smallest difference in 
improvement (i.e., -1-3.10%), whereas students with disabilities in CAG and non-CAG schools 
exhibited the largest difference in improvement (i.e., -16.6%). Black students in CAG and non- 
CAG schools exhibited the second highest difference in improvement (i.e., -15.39%). 

Closing the achievement gap in mathematics 

Figure 13 presents the mathematics aciiievement trends among all WV students in non- 
CAG schools (i.e., the remainder of the state). Results are presented by all subgroups combined 
and by the three subgroups (Black, economically disadvantaged, and students with disabilities). 
These data will serve as a point of comparison for students in CAG schools. 
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Figure 12. Trends in the Percentage of Students Scoring at Proficiency in WESTEST Mathematics 
in aii West Virginia (Non-CAG) Schools, 2004-2008 




2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 

Years 

» State {All Students) Proficiency Math 
M State SES Proficiency Math 
A State Black Proficiency Math 

Note. SES = Economically disadvantaged; SwD = Students with disabilities 



The first (top) trend line in Figure 13 indicates that statewide proficiency increased 
shghtly between 2004 and 2006 (i.e., 67.86% to 75.36%) and exhibited a plateau for years 2007 
and 2008 (i.e., 75.58% and 75.34%). Economically disadvantaged students exhibited the 
smallest gap among the subgroups when compared to the state average with a difference of 
-10.03% in 2004, which decreased to -7.81% in 2008. Bla(± students exhibited an even greater 
achievement gap in 2004 vdien compared to the state (i.e., -15.72%), vdiidi decreased to 
- 10.88% in 2008. Again, the differences in proficiency scores fiom 2004 to 2008 indicate that in 
non-CAG schools. Black students' rates of improvement exceeded that of economically 
disadvantaged students in mathematics proficiency. Students with disabilities in non-CAG 
schools exhibited the greatest achievement gap (i.e., -38.29%) in 2004, vdiich decreased to 
-33.47%. 
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Figure 13 depicts the comparison in mathematics achievement trends among all students 
in CAG schools, all students in non- GAG schools (i.e., the remainder of the state), and all of the 
subgroups of interest in the CAG schools. 



Figure 13. Trends in the Percentage of Students Scoring at Proficiency in WESTEST Mathematics, in 
CAG Schools (Subgroups and All Groups Combined) Compared with All Groups Combined 
in All West Virginia (Non-CAG) Schools 
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Note. SES = Economically disadvantaged; SwD = Students with disabilities 



The first two trend lines in Figure 13 indicate that students in CAG schools, on average, 
exhibited a -7.98% deficit in mathematics proficiency scores in 2003 compared to the state 
average-more than double the reading/ language arts defidt-^Dut closed that gap to -2.27% by 
2008. Economically disadvantaged students exhibited the smallest gap among the subgroups 
compared to the state average with a deficiency of -14.22% in 2004, vbidi decreased to -9.14% 
in 2008. Black students exhibited an even greater achievement gap in 2004 when compared to 
the state (i.e., -21.6%), vbidi decreased to - 11.19% in 2008. Again, the differences in proficiency 
scores indicate that Blac:k students' rate of improvement exceeded that of economically 
disadvantaged students in mathematics proficiency. Further, these increases in proficiency 
exceeded the rate of improvement of economically disadvantaged and Blac:k students in non- 
CAG sciiools (see Figure 9). Students with disabihties exhibited the greatest aciiievement gap 
(i.e., -42.58%) in 2004, which decreased to -0.51% in 2008. Again, students with disabilities in 
CAG sdiools outperformed non- CAG students with disabilities in proficiency score 
improvement from 2004-2008 (see Figure 14 ). 
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Figure 14 depicts the differences in mathematics proficiency improvements between all 
students in CAG and non-CAG schools, as weU as differences in proficiency improvements 
between subgroups in CAG and non-CAG subgroups of interest. 



Figure 14. Differences between WESTEST Mathematics Achievement Gains in CAG Schools (Subgroups 
and All Groups Combined) and All West Virginia (Non-CAG) Schools from 2004 to 2008 




Note. SES = Economically disadvantaged; SwD = Students with disabilities 



As was the case with reading/ language arts, the gains in CAG schools exceeded the state 
in all cases. The proficiency increase between all students in CAG and non-CAG schools differed 
by -1-5.71%; the proficiency increases among subgroups varied between -1-12.56% and -1-19.55%. 
Economically disadvantaged students in CAG and non-CAG schools exhibited the smallest 
difference in improvement (i.e., -f-3.49%), whereas students with disabilities in CAG and non- 
CAG sciiools exhibited the largest difference in improvement (i.e., -f-7.26%). Black students in 
CAG and non-CAG schools exhibited the second highest difference in improvement (i.e., 
-H5.57%). 
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Section II. Focus Group Results 



Research Question 3 

What has been the focus of the CAG haisons over the course of the project and 
what do the haisons percave the impact of the program to be? 

Following the transcription of comments coUected during the focus group sessions, the 
first stage of analysis involved reading and rereading each CAG liaison comment and assigning it 
to one of the six standards that guided the work of the CAGs, ^Ahich produced the following 
results: 

1. Bringing Focus— 36 comments 

2. Leading Change-^ comments 

3. Developing Accountabihty— 8 comments 

4. Building Capacity— 14 comments 

5. Creating Community— 11 comments 

6. Growing Professionally— 17 comments 

Comments that did not fit into this organizing schema were coded into either the 
"General Information" and "Challenges/ Successes" categories. A cursory look at the number of 
comments made about each of the standards suggests that during the 5 years of the CAG Schools 
program, the CAG haisons focused their work more stron^y on Standards 1, 4, and 6— which 
may also indicate vdiich portion of their work the CAGs valued most. 

In the second stage of analysis WVDE researchers examined comments categorized 
within each of the six standards and noted vdiich ones had ideas or phrases that were repeated 
and formed themes. Table 23 displays the four themes for each standard that had the greatest 
number of comments (24 themes total). The third stage of analysis involved collapsing the 24 
themes into five overall themes for discussion. 

Thus, M^en CAG haisons had a guided conversation concerning their roles within their 
schools and vdiat they have accomplished within 5 years, it appears that they coUectively view 
some roles and activities as being most important in their work with schools, administrators, 
teachers, and students. Their comments can be paraphrased in the following statements: 

• Their work with schools must always result in increased student learning and 
achievement. 

• Their work with teachers must instruct/ inform them in how to focus their work on 
increased student learning and achievement. 

• Their work with counties and schools must teach principals and other leaders how to 
increase open communication with aU staff members. 

• Their work with schools must result in staff members being asked their thoughts and 
having ownership of school improvement efforts. 

• Their work with teachers must instruct them in how to use a variety of assessment data 
in making decisions related to instruction and student performance. 

• Their work with teachers must facilitate all PD and include follow-up regarding use of 
that PD by teachers ^o were trained. 
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Table 23. Six Guiding Standards and the Four Major Themes Associated With Each Standard 



Standard 3 Standard 5 Standard 6 

Standard 1 Standard 2 Developing Standard 4 Creating Growing 

Bringing focus Leading change accountability Building capacity community professionally 



Act as liaison and 
provide support 
for teachers, 
schools, and 
county 
administration 


Writing school 
improvement 
plans 


Flelping school 
administration 
have better 
communication 
with staff 


Mentoring 
principals— 
leading them to 
have open 
discussions with 
staff members— 
teach them how 
to accept 
criticism— suggest 
what the next 
work should be 


Serve as 
liaison/link 
between school, 
county, and WVDE 


Present at CAG 
Leadership 
Conference; 
facilitate 
training with 
Principal 
Leadership Conf. 


Instruct in using all 
types of data— get 
teachers to focus 
on data 


Use of research- 
based strategies/ 
techniques to 
improve student 
learning 


Flelping schools 
move from using 
only WESTEST 
data to using 
formative 
assessments 
including ISS 


Building capacity 
for school 
improvement by 
increasing student 
learning 


CAGs find 
resources and 
connect those 
resources to the 
schools that need 
them 


Train teachers in 
teams and grade- 
level groups— may 
involve 3-4 
schools, but with 
small-group focus 


Focus on 
increased student 
learning/ 
achievement 


Partnership 
activities such as : 
being part of 
Management 
Team at county 
office 


Flelping schools in 
involving staff in 
ways to improve 
their schools 


Building capacity 
in schools with 
tremendous staff 
turnover— help 
remaining staff to 
sustain the 
student 
improvement that 
has been made 


CAGs guide the 
schools— not 
direct them— 
support schools in 
decisions 


CAGs follow up on 
use of PD— 
mentor by 
building in-house 
leadership in 
administration 
and staff 


Facilitate all 
teacher PD 


Becoming part of 
the collective 
school "WE" 


Flelping schools 
have common 
goals for their 
improvement 
efforts 


Work with 
superintendent- 
have access to him 
or her— take 
problems to that 
level 


Flelp schools 
develop 
ownership— 
empower 
principals 


CAG training 
involves meeting 
10 days as cadre- 
share what is 
working— do own 
PD— share 
challenges— 
training is 
responsive to our 
needs 



KEY : 1. BLUE = increased student learning and achievement 



2. ROSE = providing guidance and support to schools— not directing them 

3. LIME YELLOW = increased open communication with school staff 

4. ORANGE = CAG liaisons facilitate and foUow-up with all teacher PD 

5. TURQUOISE = focus on using data to drive instruction and assessment— aU types of data 

During the first two stages of data analysis, it became dear that some comments did not 
seem to fit neatly within any of the six standards. Those comments were then coded within two 
spedal categories. General Information and Successes/ Challenges. Thirteen comments vyere 
categorized as General Information and 35 as Successes/ Challenges. 
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CAG liaisons made many substantive comments under these two categories. For 
example, all of the CAGs noted that they had no preparation time throughout the day. All of 
their preparations to work with teachers, principals, or county administrators had to be 
completed on their own time during evenings and on weekends. This trend in CAG's comments 
suggests that they performed with a high level of dedication and commitment to their work. 



Additionally, all of the CAGs reported that there was not vdiat could be called a "typical" 
day at the schools. A day might start out working one-on-one with individual teachers on 
specific instructional strategies such as benchmarking or using techSteps, or it might involve 
having morning meetings with administrators; conducting a walk-throu^ with a principal or 
other leader; mentoring a new principal (one CAG reported working with three schools that all 
had new principals); handling e-mails or dealing with areas of frustration for teachers or 
administrators; attending a team meeting with teachers; working with the principal and others 

to prepare for special education monitoring visits; 
mentoring a principal in how to conduct classroom/ teacher 
observations, organize paperwork, or manage scheduling; 
working with professional learning communities (PLCs); 
leading a book study; working with the county to involve 
parents and community members, local churches, and 
community centers; providing training for teachers or 
following up on their use of the training materials/ 
strategies; working with teachers in understanding the 
value of a variety of data and how to use those data to drive 
instruction and student learning; and so forth. 



Some teachers ARE finding 
success in the dassroom, 
which means that some 
students ARE also finding 
success, and the schools are 
feeling better. 

—CAG liaison 



One comment induded in the General Information category was, "CAGs know they've 



made progress vhen the school's culture changes from one 
of insularity or blaming others — ^“us against them'— to one 
of openness, collaboration, and a focus on student 
achievement and solving self- identified problems." A 
second CAG liaison thought CAGs had scored a success rate 
of 8.5— 9.0, but added that there was always room for 
growth, and that teachers must learn to use data to show 
that a program or process is successful. 



This theme of changes in school culture was also 
echoed in comments categorized as Successes/ Challenges. 

For example, one CAG commented, "Maintaining a positive 
culture is hardest; it is easy for a culture to turn toxic." One 
CAG agreed and provided the example of an elementaiy 
school in one West Virginia county. He stated that the 
school at one time had been a nice community school, but school consolidation changed the 
demographics ovemi^t and with that change the attitudes of some teachers decfined. The 
school had been in an upper middle class neighborhood, but then low- income housing projects 
were introduced into the area; hi^- quality teachers began to leave and there was a genuine 



CAGs know they've made 
progress when the school's 
culture changes from one of 
insularity or blaming 
others —us against them— 
to one of openness, 
collaboration, and focus on 
student achievement. 



—CAG liaison 
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concern over the exodus of these teachers. The principal, vdio was near retirement, expected the 
teachers to become a learning community, but he was not willing to push the envelope in order 
to make that happen. Now, that same school is one of the CAG schools' biggest successes. 
Recently, the WESTEST results came back with Black subgroup students outscoring White 
subgroup students. This was a great accomphshment in the 3rd 
year of the CAG liaison introduction into the school, and it was a 
direct result of that work with the staff and principal according to 
the CAG haisons. 

The challenge of maintaining or creating a positive school 
culture was echoed by other CAGs as well, but others reported that 
the cultures of the schools had improved over the last 5 years. In 
the past, there were problems such as teachers yelling at their 
students; the school building not being clean; the principal seldom 
leaving his or her office; administration never checking on the 
restrooms; lack of communication from administration to the teachers; no morning 
annormcements; teachers only receiving directives. This has changed to more open and dear 
communication. 

CAGs pointed out that there are tremendous pressures on public schools today and schools 
often find httle consistency in ^at they are recjuired to do. The CAG liaisons try to recondle or 
facilitate understanding of all of the information coming from the counties and state for the 
schools. One CAG commented on the difficulty of recondling all the requirements of the state 
while still meeting the needs of children. Also, people feel pressure about how to get everything 
done with too little time in vdiicii to do it. 

Along this same line, two CAGs reported feeling that their biggest success was that their 
sciiools were operating in the belief that it doesn't matter ^diat is happening out there (within 
state, central office, etc.), but instead it mattered vriiat the children needed to learn. One CAG 
haison stated, "Some teachers ARE finding success in the dassroom, vdiidi means that some 
students are also finding success, and the sciiools are feeling better." 

In addition, several CAG liaisons reported that now schools and teadiers are not making 
so many excuses for how thdr students are performing. Instead, the focus is more about how 
children can get the information/ knovdedge they need and how teachers can help children 
learn. In general, the CAGs seemed to think that each school was getting to the point that the 
important thing for them was that their students were all learners and that they continued to 
learn. 

When they were asked how positive change had been fadhtated in their schools, the 
CAGs responded with the following points: 

• Talking about it 

• T alking about improvement of instruction 

• Using the process as instructed 

• Networking between schools 



Maintaining a 
positive culture is 
hardest; it is easy 
for a culture to turn 
toxic. 

—CAG liaison 
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• Attitude changes 

• Cultural changes 

When the CAG liaisons were asked about possible challenges or barriers to their work 
with schools, they responded with the following points: 

• Getting in the door 

• Time is a huge factor 

• Money is an issue 

• Lack of leadership or resistance to leadership 

• Support can be a maj or barrier 

• Retirements 

• Challenge of maintaining a positive culture 

• Multiple issues 

Despite the challenges and barriers to their work, the CAGs overall thought they had 
made some important accomplishments over the last 5 years. CAGs reported having moved their 
schools from lecture-based teaching methods and models to other types of instructional 
methods. Additionally, CAGs reported making improvements in instructional teams by changing 
the way that meetings were conducted. Finally, the CAGs noted that they had increased the use 
of classroom assessments for learning, and that they had improved school collaborative 
relationships. 



Section III. Survey Results 



Research Question 4 

How do CAG schools compare with high performing learning communities in 
terms of continuous school improvement practices? 

As mentioned in the Research Methodology section of this report, to investigate RQ4, the 
WVDE researchers administered the Continuous School Improvement Questionnaire ([CSIQ] 
Meehan, et al., 2006). The CSIQ is a measure of schools' performance on seven dimensions 
critical to school improvement and contains 70 Likert- type response format items, vdiich 
comprise seven distinct subscales: (a) Learning Culture, (b) School- Farnily- Community 
Connections, (c) Shared Leadership, (d) Shared Goals for Learning, (e) Purposeftil Student 
Assessment, (f) Effective Teaching, and (g) Ahgned and Balanced Curriculum. Scores on the 
survey are measured against other normative schools with similar characteristics (average 
schools), and against "high performing learning communities" (HPLC schools), vdiich are 
schools that have received national recognition as schools of excellence, among other criteria 
that indicate commitment to continuous learning and improvement (Meehan et al., 2006). The 
survey was administered in CAG schools in 2006 at the outset of the project by an external 
contractor, Edvantia, Inc. WVDE researchers re- administered the instruments in 2010 as part of 
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the current evaluation. The following section presents the results of the 2010 administration of 
the CSIQ. 

Mean CSIQ scores for CAG schools 

Descriptive data from the CSIQ are presented in the tables below disaggregated by 
programmatic level. It should be noted that the majority of CAG schools were elementary 
schools (n = 13). Fewer were middle schools (n =9) and hi^ schools (n =3). An additional 
school was identified as having an "other" configuration for the purposes of normative 
comparisons. This school was a PreK- 8 configured school. 

Elementary schools 

In elementary schools, the mean score for CAG schools was consistently higher than the 
mean score for average schools in the norm-referenced sample. This was true for all seven 
subscales. In aU subscales, however, the mean scores for CAG elementary schools were lower 
than the mean scores for norm- referenced elementary schools identified as HPLC schools (see 
Table 24). 

Table 24. Overall CSIQ Findings for CAG Elementary Schools, 2010. 



Mean scores 



Subscale 


CAG 

schools 


HPLC 

schools 


Average 

schools 


Conclusions 


Learning culture 


49.69 


51.25 


47.28 


CAG schools above average but 
below HPLC mean. 


School-family-community 

connections 


46.19 


52.46 


45.07 


CAG schools above average and 
below HPLC mean. 


Shared leadership 


48.11 


51.64 


45.56 


CAG schools above average but 
below HPLC mean. 


Shared Goals for Learning 


49.48 


53.25 


47.81 


CAG schools above average but 
below HPLC mean. 


Purposeful student 
assessment 


49.43 


53.14 


47.16 


CAG schools above average but 
below HPLC mean. 


Effective teaching 


49.59 


53.08 


48.89 


CAG schools above average but 
below HPLC mean. 


Aligned and Balanced 
Curriculum 


47.11 


49.81 


43.58 


CAG schools above average but 
below HPLC mean. 



Middle schools 

At the middle school level, the mean score for CAG schools was consistently higher than 
the mean score for average schools in the norm- referenced sample. This was true for all seven 
subscales. In all subscales, however, the mean scores for CAG middle schools were lower than 
the mean scores for norm- referenced middle schools identified as HPLCfe (see Table 25). 
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Table 25. Overall CSIQ Findings for CAG Middle Schools, 2010 



Mean scores 



Subscale 


CAG schools 


HPLC schools 


Average 

schools 


Conclusions 


Learning culture 


47.00 


50.14 


45.79 


CAG schools above average but 
below HPLC mean. 


School-family-community 

connections 


40.21 


48.29 


41.70 


CAG schools below average and 
below HPLC mean. 


Shared leadership 


44.27 


50.76 


43.94 


CAG schools above average but 
below HPLC mean. 


Shared Goals for 
Learning 


46.02 


51.27 


45.21 


CAG schools above average but 
below HPLC mean. 


Purposeful student 
assessment 


45.19 


50.88 


44.53 


CAG schools above average but 
below HPLC mean. 


Effective teaching 


46.89 


50.77 


46.35 


CAG schools above average but 
below HPLC mean. 


Aligned and Balanced 
Curriculum 


42.18 


48.29 


41.70 


CAG schools above average but 
below HPLC mean. 



High schools 

In hi^ schools, the mean score for CAG schools was consistently lower than the mean 
score for both HPLCfe and average schools in the norm- referenced sample (Table 26). In other 
words, CAG high schools included in the sample (n =2) were falling short of the benchmarks set 
by both average hi^ schools and those identified as HPLCfe. It should be noted that the small 
number of high schools in the CAG sample could bias the mean score, if those schools are 
particularly low performing. 

Table 26. Overall CSIQ Findings for CAG High Schools, 2010 

Mean scores 

Average 



Subscale 


CAG schools 


HPLC schools 


schools 


Conclusions 


Learning culture 


41.69 


49.22 


44.40 


CAG schools below average and 
below HPLC mean. 


School-family-community 

connections 


33.54 


49.01 


40.88 


CAG schools below average and 
below HPLC mean. 


Shared leadership 


37.04 


50.32 


42.71 


CAG schools below average and 
below HPLC mean. 


Shared Goals for Learning 


41.26 


49.82 


43.80 


CAG schools below average and 
below HPLC mean. 


Purposeful student 
assessment 


40.49 


49.27 


42.52 


CAG schools below average and 
below HPLC mean. 


Effective teaching 


42.03 


51.22 


45.55 


CAG schools below average and 
below HPLC mean. 


Aligned and Balanced 
Curriculum 


37.95 


47.46 


39.80 


CAG schools below average and 
below HPLC mean. 
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Other schools 

There was only one CAG school identified as having an other grade configuration. This 
school served students in grades Pre/K-8. The results of the CSIQ for this school indicated that 
the school was well above average schools in the norm- referenced sample (Table 27) . There were 
no HPLCs in the norm-referenced sample to serve as a comparison for this GAG school. It 
should be noted that these results could be biased if the single CAG school in this sample is 
particularly high performing. 

Table 27. Overall CSIQ Findings for Other School (PreK through Grade 8), 2010 



Mean Scores 



Subscale 


CAG 

schools 


HPLC 

schools 


Average 

schools 


Conclusions 


Learning culture 


53.25 


* 


45.10 


CAG schools above average mean. 


School-family-community 

connections 


53.62 


* 


40.92 


CAG schools above average mean. 


Shared leadership 


52.25 


* 


44.94 


CAG schools above average mean. 


Shared Goals for Learning 


54.62 


* 


45.27 


CAG schools above average mean. 


Purposeful student assessment 


55.37 


* 


43.65 


CAG schools above average mean. 


Effective teaching 


53.62 


* 


47.14 


CAG schools above average mean. 


Aligned and Balanced Curriculum 


54.86 


* 


40.17 


CAG schools above average mean. 



* No norm-referenced data were available for "other" configuration HPLCs. 

Norm-referenced CSIQ results for CAG schools 



Figure 15 through Figure 21 illustrate the percentile rankings for each of the 26 CAG 
schools for each of the seven subscales (learning culture, school- family- community connections, 
etc.), vdiich, as in Table 24 through Table 27 above, are derived by comparing the mean subscale 
score for each CAG school with data from a larger group of norm- referenced schools in the CSIQ 
User Manual and Technical Report (Meehan, et al., 2006). For this analysis, WVDE researchers 
compared each CAG school to the norm for Average (not HPLC) schools of the same 
programmatic level (e.g, elementary, middle, high, and other) on each of the subscales. Each 
bar represents the percentile rank for an individual CAG school. The line that crosses the bars 
on each chart represents the median percentile rank for all CAG schools. The median percentile 
serves as a summary measure of the rank of the CAG schools vdien they are aggregated into a 
sin^e group. Interpretation is relatively straightforward. For example, if the median for CAG 
schools falls below the 50*^ percentile, this would mean that the CAG schools exhibited a CSIQ 
subscale score that was lower than 50% of the average norm- referenced schools.^ 



5 All schools' identities were masked for these analyses. 
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Learning culture 

The median percentile rank of GAG schools on the Learning Culture subscale was 71. In 
other words, only 29% of norm- referenced average schools exhibited scores on the subscale that 
were higher than the median score of the GAG schools at the end of the 5-year program (Figure 
15) . Learning culture was the second- highest rated of the CSIQ subscales, indicating it as an area 
of strength for GAG schools. With respect to individual results. School Q exhibited the lowest 
percentile rank on the subscale, scoring lower than 97% of norm- referenced schools. School J 
exhibited the hipest rank, with only 1% of norm- referenced schools scoring higher. 



Figure 15. Learning Culture— CSIQ Percentile Ranking of Individual CAG Schools on Norm-Referenced 
Scale (Average Schools), and Median Score for All CAG Schools, 2010 
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School-family-community connections 

The median percentile rank of CAG schools on the School- Farmly-Ctormnunity 
Ctonnections subscale was 43. In other words, 57% of norm- referenced average schools exhibited 
scores on the subscale that were higher than the median score of the CAG schools at the end of 
the 5-year program (Figure 16). This was the lowest rated of the seven CSIQ subscales, 
indicating it as a critical area of concern for CAG schools. With respect to individual school 
results. School P exhibited the lowest percentile rank on the subscale, scoring lower than 99% of 
norm- referenced schools. School L exhibited the highest rank, outscoring all other norm- 
referenced schools. 
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Figure 16. School-Family-Community Connections— CSIQ Percentile Ranking of Individual CAG 
Schools on Norm-Referenced Scale (Average Schools), and Median Score for All CAG 
Schools, 2010 
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Shared leadership 

The median percentile rank of CAG schools on the Shared Leadership subscale was 66. 
In other words, only 34% of norm- referenced schools exhibited scores on the subscale that were 
higher than the median score of the CAG schools at the end of the 5-year program, indicating 
this to be an area of somewhat less concern than others for CAG schools (Figure 17). With 
respect to individual school results. School Q exhibited the lowest percentile rank on the 
subscale, scoring lower than 96% of norm- referenced schools. Schools L and S exhibited the 
hipest ranks, with only 7% of norm- referenced schools scoring hi^er. 

Shared goals for learning 

The median percentile rank of CAG schools on the Shared Goals for Learning subscale 
was 66 (Figure 18). In other words, only 34% of norm- referenced average schools exhibited 
scores on the subscale that were hi^er than the median score of the CAG schools at the end of 
the 5-year program, indicating it to be an area of somevdiat less concern than others for CAG 
schools. School P exhibited the lowest percentile rank on the subscale, scoring lower than 95% of 
norm- referenced schools. School J exhibited the highest rank, with only 1% of norm- referenced 
schools scoring higher. 
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Figure 17. Shared Leadership— CSIQ Percentile Ranking of Individual CAG Schools on Norm- 
Referenced Scale (Average Schools), and Median Score for All CAG Schools, 2010 
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Figure 18. Shared Goals for Learning— CSIQ Percentile Ranking of Individual CAG Schools on Norm- 
Referenced Scale (Average Schools), and Median Score for All CAG Schools, 2010 
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Purposeful student assessment 

The median percentile rank of CAG schools on the Purposeful Student Assessment 
subscale was 70. In other words, only 30% of norm- referenced average schools exhibited scores 
on the subscale that were higher than the median score of the GAG schools at the end of the 5- 
year program, indicating this to be an area of strength for GAG schools (Figure 19). With respect 
to individual school results. School Q exhibited the lowest percentile rank on the subscale, 
scoring lower than 93% of norm- referenced schools. School J exhibited the highest rank, with 
only 1% of norm- referenced schools scoring higher. 



Figure 19. Purposeful Student Assessment— CSIQ Percentile Ranking of Individual CAG Schools 
on Norm-Referenced Scale (Average Schools), and Median Score for All CAG Schools, 
2010 
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Effective teaching 

The median percentile rank of GAG schools on the E ffective Teaching subscale was 55. In 
other words, 45% of norm- referenced average schools exhibited scores on the subscale that were 
hi^er than the median score of the CAG schools at the end of the 5-year program (Figure 20). 
This was the second lowest rated of the CSIQ subscales, indicating it is an area of concern for 
CAG schools. With respect to individual school results. School Q exhibited the lowest percentile 
rank on the subscale, scoring lower than 99% of norm- referenced schools. School J exhibited the 
hipest rank, with only 1% of norm- referenced schools scoring higher. 
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Figure 20. Effective Teaching— CSIQ Percentile Ranking of Individual CAG Schools on Norm- 
Referenced Scale (Average Schools), and Median Score for All CAG Schools, 2010 
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Aligned and balanced curriculum 

The median percentile rank of CAG schools on the Aligned and Balanced CXirriculmn 
subscale was 76. In other words, only 24% of norm- referenced average schools exhibited scores 
on the subscale that were higher than the median score of the CAG schools at the end of the 5- 
year program (Figure 21) . This was the highest rated of the CSIQ subscales, indicating it to be an 
area of particular strength for CAG schools. With respect to individual school results. School Q 
exhibited the lowest percentile rank on the subscale, scoring lower than 92% of norm- referenced 
schools. Schools J and L exhibited the highest ranks, with only 1% of norm- referenced schools 
scoring hi^er. 
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Figure 21. Aligned and Balanced Curriculum— CSIQ Percentile Ranking of Individual CAG Schools 
on Norm-Referenced Scale (Average Schools), and Median Score for All CAG Schools, 
2010 
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Summary of the CSIQ Findings 

One overardiing caveat must be expressed prior to interpreting these results. These 
findings are descriptive in nature. While it is interesting and useful to compare the results of 
CAG schools with norm-referenced schools, there is not sufficient evidence to determine 
vdiether these differences between CAG schools and norm- referenced schools were 
meaningfully and statistically significant. Nor does this study design allow us to determine if the 
survey results are directly attributable to the presence of the CAG program. With these 
limitations in mind, the following interpretations of the CSIQ findings are offered: 

• In general, elementary and middle schools served by the CAG program appear to have 
performed hi^er on the CSIQ than high schools. Elementary schools outperformed 
average norm- referenced schools on all seven subscales, and middle schools 
outperformed average schools on all but one subscale. While this finding cannot be 
attributed directly to the CAG program, this is an indicator of success for these schools. 

• High schools served hy the CAG program were outperformed by average norm- 
referenced schools and schools identified as high performing learning communities 
(HPLCs) on all seven subscales. This indicates that the CAG high schools continue to 
exhibit a strong need for intervention. 

• In no case did CAG schools outperform norm- referenced schools that were identified as 
HPLCs. This is not particularly surprising given the fact that aU CAG schools were 
identified as requiring substantial intervention prior to being included in the CAG 
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program. However, CAG schools should strive to reach the benchmarks set by HPLCs, 
and in some cases the CAG schools' scores did approach those of HPLCs. 

• The CSIQ identified at least two critical areas of need for CAG schools as a group. These 
include building school- fanuly-cornmunity connections and increasing school- wide 
capacity for effective teaching. These were both areas where the median for CAG schools 
fell below the 50*^ percentile. These areas are absolutely critical to school improvement, 
and as such, it is recommended that WVDE continue to focus professional development 
and other resources on addressing these deficiencies in CAG schools. 

• The CSIQ identified at least three areas of strength for CAG schools as a group. These 
include the presence of an aligned and balanced curriculum, a positive learning culture, 
and hi^ quality and purposeful student assessment. These are areas ^ere the median 
for CAG schools fell above the 70*^ percentile for norm- referenced average schools. As 
such, these areas appear to be well- covered in most CAG schools. However, there were 
some specific CAG schools Mfiere the percentile ranks for these subscales were 
particularly low. Given thar results on the remaining subscales, it is likely that these 
schools still require substantial intervention. 

Research Question 5 

What are the areas of strength and concern in CAG schools in terms of school 
culture and capacity for improvement? 

To study this question, two additional surveys were administered in all 26 GAG schools 
in the spring of 2010. To address the school culture aspect of the question, the Perceptions of 
School Culture questionnaire ([POSC] Covdey, et al., 2006) was administered; likewise, to 
investigate schools' capacity to improve, the Measure of School Capacity for Improvement 
([MSQ] Hu^es, et al., 2006) was administered. Results and discussion of the POSC survey are 
presented first, followed by results of the MSQ . 

Perceptions of School Culture survey results 

The following section presents the results of the 2010 administration of the POSC survey 
in CAG schools. As described earlier, the POSC is a tool to measure perceptions of faculty and 
staff about six key components of school culture: (a) Collaborative Working Relationships, (b) 
Student- Centered Vision, Mission, and Pohdes, (c) Student Responsibihty for Learning, (d) 
Teacher Responsibility for Learning, (e) Inviting Physical Environment, and (f) Students and 
Parents as Decision Makers. The POSC survey was completed by 799 faculty and staff members 
in all 26 CAG schools. WVDE researchers also compared the average POSC subscale scores for 
the 26 CAG schools to the mean subscale scores for all CAG schools as reported in a 2006 
Edvantia research report (Gilchrist, et al., 2006). It should be noted that these comparisons 
were made solely for descriptive purposes. Additionally, it must be noted that fewer CAG schools 
completed the POSC instrument in 2010 than in 2005 (i.e., 26 schools in 2010 cf. 30 schools in 
2005). Therefore, no conclusions can be made as to Mfiether the POSC results are statistically 
significant or directly attributable to the presence of the CAG program. 
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Comparison of 2006 and 2010 mean POSC subscale scores 



Figure 22 below presents the mean scores for CAG schools on each subscale of the POSC 
instrument. The blue bars represent the mean scores for the original 30 GAG schools as 
pubhshed in the 2006 Edvantia report (Gilchrist, et al., 2006). The green bars represents the 
mean scores for the 26 current CAG schools as collected for this report in 20 10 . 



Figure 22. Comparison of 2005 and 2010 Mean Scores for All CAG Schools on Six POSC Subscales 
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It is evident from Figure 22 that, with the exception of two subscales (Collaborative 
Working Relationships and Student Responsibihty for Learning), the POSC scores were higher 
for CAG schools in 2010 than in 2006. Table 28 displays means, standard deviations, and the 
effect size of the change (Cohen's d) from the 2006 administration of the POSC to the 2010 
administration. A Cohen's d of .20 is considered a small effect, so with values ranging from - .11 
to. 12, the school culture effects shown here appear to be quite small. 
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Table 28. POSC 2005 and 2010 Means and Standard Deviations, and Effect Size of Change (2005-2010) by 
Subscale 



Subscale 


Mean 

2006 

(N=28) 


Standard 

deviation 

2006 


Mean 

2010 

(N=26) 


Standard 

deviation 

2010 


Effect Size 
of change 
(Cohen's d) 


Sl-Collaborative Working Relationships 


50.24 


10.39 


49.03 


10.58 


-.11 


S2-Student-Centered Vision, Mission and Policies 


53.00 


8.96 


54.09 


8.73 


.12 


S3-Student Responsibility for Learning 


41.60 


9.33 


40.88 


9.61 


-.07 


S4-Teacher Responsibility for Learning 


50.91 


8.75 


51.37 


8.71 


.05 


S5-lnviting Physical Environment 


19.41 


4.56 


19.82 


4.03 


.09 


S6 -Students and Parents as Decision Makers 


15.67 


3.86 


15.74 


3.83 


.02 



Norm-referenced POSC results for CAG schools 

Figures 17- 22 illustrate the percentile rankings for each of the 26 CAG schools. These 
percentile rankings are derived by comparing the mean subscale score for each CAG school with 
data from a larger group of norm-referenced schools in the POSC User Manual and Technical 
Report (Covdey, et al., 2006). For this analysis, WVDE researchers compared each CAG school 
to schools of the same programmatic level (e.g., elementary, middle, high, etc.). Each bar 
represents the percentile rank for an individual GAG school. The line that crosses the bars on 
each chart represents the median percentile rank for all GAG schools. The median percentile 
serves as a summary measure of the rank of the CAG schools rA^en they are aggregated into a 
single group. Interpretation is relatively strai^tforward. For example, if the median for CAG 
schools falls below the 50*^ percentile, this would mean that the GAG schools exhibited a POSC 
subscale score that was lower than 50% of the norm- referenced schools for AAtoich data were 
compiled in developing the POSC. 

Student-centered vision, mission, and policies 

The median percentile rank of CAG schools on this subscale was 82 (Figure 24). In other 
words, only 18% of norm- referenced schools exhibited average scores on this subscale that were 
higher than the median score of the CAG schools. This was the highest rated of all the POSC 
subscales, indicating it to be an area of particular strength for CAG schools. With respect to 
individual school results. School K exhibited the lowest percentile rank on the subscale, scoring 
lower than 94% of norm- referenced schools. Schools J , L and O exhibited the highest ranks, 
with only 1% of norm- referenced schools scoring higher. 

Collaborative working relationships 

The median percentile rank of CAG schools on this subscale was 64.5 (Figure 23). In 
other words, only 35.5% of norm- referenced schools exhibited average scores on the subscale 
that were higher than the average score of the CAG schools, indicating it to be an area of 
somevdiat less concern than others for CAG schools. With respect to individual school results. 
School Q exhibited the lowest percentile rank on the subscale, scoring lower than 94% of norm- 
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referenced schools. School O exhibited the highest rank, with only 1% of norm- referenced 
schools scoring higher. 



Figure 24. Student-Centered Vision, Mission, and Policies— Percentile Ranking of Individual CAG Schools 
on POSC Norm-Referenced Scale, and Median Score for All CAG Schools, 2010 
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Figure 23. Collaborative Working Relationships— Percentile Ranking of Individual CAG Schools on 
POSC Norm-Referenced Scale, and Median Score for All CAG Schools, 2010 
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Student responsibility for learning 

The median percentile rank of GAG schools on this subscale was 48.5 (Figure 25). In 
other words, approximately 51.5% of norm- referenced schools exhibited scores on this subscale 
that were hi^er than the scores of the GAG schools. This was the lowest rated of the six POSG 
subscales, indicating it to be an area of particular concern for GAG schools. With respect to 
specific school results. School X exhibited the lowest percentile rank on the subscale, scoring 
lower than 98% of norm- referenced schools. Schools J , L and O again exhibited the hipest 
ranks, with only 1% of norm- referenced schools scoring higher. 



Figure 25. Student Responsibility for Learning— Percentile Ranking of Individual CAG Schools on POSC 
Norm-Referenced Scale, and Median Score for All CAG Schools, 2010 




Median 

Teacher responsibility for learning 

The median percentile rank of GAG schools on this subscale was 75 (Figure 26). In other 
words, as an aggregated group, only 25% of norm- referenced schools exhibited scores on this 
subscale that were higher than the scores of the GAG schools. This was the second- hipest rated 
of the six POSG subscales, indicating it to be an area of strength for GAG schools. With respect to 
individual school data. School Q exhibited the lowest percentile rank on the subscale, scoring 
lower than 99% of norm- referenced schools. Schools J , L, O, and U exhibited the highest ranks, 
with only 1% of norm- referenced schools scoring higher. 
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Figure 26. Teacher Responsibility for Learning— Percentile Ranking of Individual CAG Schools on POSC 
Norm-Referenced Scale, and Median Score for All CAG Schools, 2010 
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Inviting physical environment 

The median percentile rank of CAG schools on this subscale was 52 (Figure 27). In other 
words, approximately 48% of norm-referenced schools exhibited scores on this subscale that 
were higher than the scores of the CAG schools. This was the second lowest rated of the six 
POSC subscales, indicating it to be an area of concern for CAG schools. With respect to specific 
schools. School K exhibited the lowest percentile rank on the subscale, scoring lower than 97% 
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of nomi- referenced schools. Schools J, L, and O exhibited the hipest ranks, with only 1% of 
norm- referenced schools scoring higher. 

Students and parents as decision makers 

The median percentile rank of CAG schools on this subscale was 65 (Figure 28). In other 
words, only 35% of norm- referenced schools exhibited scores on this subscale that were hi^er 
than the scores of the CAG schools, indicating it to be an area of somevdiat less concern than 
others for CAG schools. With respect to individual schools. School Q exhibited the lowest 
percentile rank on the subscale, scoring lower than 92% of norm- referenced schools. Schools J 
and O exhibited the hipest ranks, with only 1% of norm- referenced schools scoring hi^er. 



Figure 28. Students and Parents as Decision Makers— Percentile Ranking of Individual CAG Schools on 
POSC Norm-Referenced Scale, and Median Score for All CAG Schools, 2010 
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Summary of the POSC survey findings 

One important caveat must be expressed prior to interpreting these results. These 
findings are descriptive in nature. While it is interesting and useful to compare the results of 
CAG schools with norm- referenced schools, there is not sufficient evidence to determine 
whether these differences between CAG schools and norm- referenced schools were 
meaningfully or statistically significant. Nor is the study design sufficient to determine if the 
survey results are directly attributable to the presence of the CAG program. With these 
limitations in mind, the following interpretations of the POSC findings are offered: 

• The POSC identified at least two potential areas of need for the aggregated group of CAG 
schools. These include building a sense of student responsibility for learning and 
ensuring the presences of an inviting physical environment. The median percentile for 
CAG schools fell near or below the 50th percentile for both areas. Increasing students' 
sense of responsibility for their own learning, while a very difficult task, would likely 
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contribute heavily toward improving achievement in these schools. Likewise the 
presence of a positive and safe learning environment is critically important in increasing 
students' abihty to learn. If the CAG program continues, these two areas would be ideal 
places to focus resources in CAG schools. 

• The POSC identified at least two areas of strength for CAG schools. These include the 
presence of a strong Student- Centered Vision, Mission, and Pohdes and hi^ levels of 
Teacher Responsibility for Learning. For both areas the median fell above the 70*^ 
percentile in CAG schools. A strong student- centered vision is the central support 
structure for a successful school. Likewise vdien teachers possess a sense of 
responsibility for their students' learning, they are more driven to be successful with 
those students. Both areas are essential for school improvement and the CAG program 
should capitahze on these assets. 

• CAG schools exhibited a small increase on five of the seven POSC subscales during the 
course of the CAG program. The data provided in the original 2006 Edvantia report 
(Gilchrist, et al., 2006) were not sufficient to allow WVDE to make any conclusions 
about the statistical significance of these changes. However, the effect size statistics 
calculated for each change indicated that the changes were ne^gible, at best. 
Additionally, the retrospective nature of this study does not allow WVDE to isolate the 
CAG program as the agent driving this change. However, these data do illustrate, on a 
descriptive level that at least some positive cultural change has taken place in CAG 
schools over the course of the project. 

Measure of School Capacity for Improvement results 

The following section presents the results of the 2010 administration of the MSQ survey 
(Hu^es, et al., 2006), beginning with a comparison between these results and results obtained 
in a 2006 administration of the survey CAG schools. As mentioned earher, the MSQ contains 58 
Likert-type response format items vdiich comprise seven distinct subscales: (a) Equity in 
Practice, (b) Expectations for Student Performance, (c) Differentiated Instruction, (d) 
Improvement Program Coherence, (e) Peer Reviewed Practice, (f) Coordinated Curriculum, and 
(g) Technical Resources. The MSQ was completed by 784 faculty and staff members (N =784) 
in 26 CAG schools of various programmatic levels. 

Comparison of 2006 and 2010 mean MSCI subscale scores 

Figure 29 below presents the mean scores for CAG schools on each subscale of the MSQ 
instrument. The orange bars represent the mean scores for the original 30 CAG schools as 
published in the 2006 Edvantia report: Baseline Study of Selected Professional Development 
Schools in West Virginia (Gilchrist, et al., 2006). The red bar represents the mean score for the 
26 current CAG schools as collected for the purposes of this report in 2010. 
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Figure 29. Comparison of 2005 and 2010 Mean Scores for All CAG Schools on Seven MSCI Subscales 
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It is evident from the figure that, for five of the seven MSQ subscales, mean scores were 
shghtly higher for CAG schools in 2010 than in 2006. Scores were lower in 2010 for the Equity 
in Practices subscale (SI) and stayed the same for the Peer Reviewed Practice subscale (S5). It 
should be noted that these findings are descriptive in nature; the data collected are not sufficient 
for WVDE researchers to make any claims of statistical significance or direct attribution to the 
CAG program. Additionally, it should be noted that fewer CAG schools completed the MSQ 
instrument in 2010 than in 2005 (i.e., 26 schools in 2010 cf. 30 schools in 2006). Table 29 
displays means, standard deviations, and the effect size of the change (Cohen's d) from the 2005 
administration of the POSC to the 2010 administration. A Cohen's d of .20 is considered a small 
effect, so with values ranging from - .012 to .21, the school climate effects shown here are quite 
small. 
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Table 29. MSCI 2005 and 2010 Means and Standard Deviations, and Effect Size of Change (2005-2010) by 



Subscale 


MSCI Subscale 


Mean 

(N=29) 

2006 


SD 

2006 


Mean 

(N=26) 

2010 


SD 

2010 


Effect size of 
change 
(Cohen's d) 


Si-Equity in Practice 


4.98 


.67 


4.89 


.84 


-.012 


S2-Expectations for Student Performance 


4.55 


.85 


4.57 


.87 


.02 


SS-Differentiated Instruction 


4.64 


.85 


4.69 


.84 


.06 


S4-lmprovement Program Coherence 


4.29 


.83 


4.40 


.83 


.13 


S5-Peer Reviewed Practice 


3.42 


1.38 


3.42 


1.40 


.0 


S6-Coordinated Curriculum 


4.13 


1.13 


4.36 


1.03 


.21 


S7-Technical Resources 


4.42 


.10 


4.50 


.96 


.12 



Norm-referenced MSCI results for CAG schools 

Figure 31 through Figure 36 illustrate the percentile rankings for each of the 26 CAG 
schools. These percentile ranks are derived by comparing the mean subscale score for each CAG 
school with data from a larger group of norm- referenced schools in the MSQ User Manual and 
Technical Report (Hughes, et al., 2006). For this analysis, WVDE researchers compared each 
GAG school to schools of the same programmatic level (e.g., elementary, middle, high, etc.). 
Each bar represents the percentile rank for an individual CAG school. The line that crosses the 
bars on each chart represents the median percentile rank for all CAG schools. The median 
percentile serves as a summary measure of the rank of the CAG schools vhen they are 
aggregated into a sin^e group. Interpretation is relatively straightforward. For example, if the 
median for CAG schools falls below the 50th percentile, this would mean that the CAG schools 
exhibited a MSQ subscale score that was lower than 50% of the norm- referenced schools. 

Expectations for student performance 

The median percentile rank of CAG schools on this subscale was 50 (Figure 31). In other 
words, 50% of norm- referenced schools exhibited scores on the subscale that were higher than 
the scores of the CAG schools. This was one of the lowest rated of the MSQ subscales, indicating 
it as an area of concern for CAG schools. With respect to individual results. Schools O and Q 
exhibited the lowest percentile ranks on the subscale, scoring lower than 97% of norm- 
referenced schools. School J exhibited the hipest rank, with only 10% of norm-referenced 
schools scoring higher. 

Equity in practice 

The median percentile rank of CAG schools on this subscale was 55 (Figure 30). In other 
words, 45% of norm- referenced schools exhibited scores on the subscale that were higher than 
the scores of the CAG schools. This was the second- lowest rated of the MSQ subscales, 
indicating it as a potential area of concern for CAG schools. With respect to individual results. 
School Q exhibited the lowest percentile rank on the subscale, scoring lower than 97% of norm- 
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referenced schools. Schools J and V exhibited the highest ranks, with only 1% of norm- 
referenced schools scoring higher. 



Figure 31. Expectations for Student Performance— Percentile Ranking of Individual CAG Schools on MSCI 
Norm-Referenced Scale, and Median Score for All CAG Schools, 2010 
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Figure 30. Equity in Practice— Percentile Ranking of Individual CAG Schools on MSCI Norm-Referenced 
Scale, and Median Score for All CAG Schools, 2010 
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Differentiated instruction 



The median percentile rank of CAG schools on this subscale was 76 (Figure 32). In other 
words, only 24% of norm- referenced schools exhibited scores on this subscale that were higher 
than the scores of the GAG schools. This was the hipest rated of the MSQ subscales, indicating 
it as an area of particular strength for GAG schools. With respect to individual results. School Q 
exhibited the lowest percentile rank on the subscale, scoring lower than 96% of norm- referenced 
schools. School J exhibited the hipest rank, with only 2% of norm- referenced schools scoring 
hi^er. 



Figure 32. Differentiated Instruction— Percentile Ranking of Individual CAG Schools on MSCI Norm- 
Referenced Scale, and Median Score for All CAG Schools, 2010 
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Improvement program coherence 



The median percentile rank of CAG schools on this subscale was 58 (Figure 33). In other 
words, 42% of norm-referenced schools exhibited average scores on the subscale that were 
hi^er than the average scores of the CAG schools. This was the third- lowest rated of the MSQ 
subscales, indicating it as a potential area of concern for CAG schools. With respect to individual 
results. School Q exhibited the lowest percentile rank on the subscale, scoring lower than 92% of 
norm- referenced schools. School S exhibited the hipest rank, with only 1% of norm- referenced 
schools scoring higher. 
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Figure 33. Improvement Program Coherence— Percentile Ranking of Individual CAG Schools on MSCI 
Norm-Referenced Scale, and Median Score for All CAG Schools, 2010 
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Peer reviewed practice 

The median percentile rank of CAG schools on this subscale was 50 (Figure 34). In other 
words, 50% of norm- referenced schools exhibited scores on the subscale that were higher than 
the scores of the CAG schools. This was among the lowest rated of the MSCI subscales, 
indicating it as an area of concern for CAG schools. With respect to individual results. School Y 
exhibited the lowest percentile rank on the subscale, scoring lower than 98% of norm- referenced 



Figure 34. Peer Reviewed Practice— Percentile Ranking of Individual CAG Schools on MSCI Norm- 
Referenced Scale, and Median Score for All CAG Schools, 2010 
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schools. Schools E and S exhibited the highest ranks, with only 1% of norm- referenced schools 
scoring hi^er. 

Equity in practice 

The median percentile rank of GAG schools on this subscale was 68 (Figure 35). In other 
words, 32% of norm- referenced schools exhibited scores on the subscale that were higher than 
the scores of the GAG schools. This was the third- highest rated of the MSQ subscales, indicating 
it as a potential area of strength for GAG schools. With respect to individual results. School M 
exhibited the lowest percentile rank on the subscale, scoring lower than 88% of norm- referenced 
schools. School J exhibited the hipest rank, with only 2% of norm- referenced schools scoring 
hi^er. 



Figure 35. Equity in Practice— Percentile Ranking of Individual CAG Schools on MSCI Norm-Referenced 
Scale, and Median Score for All CAG Schools, 2010 
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Technical resources 

The median percentile rank of CAG schools on this subscale was 69 (Figure 36). In other 
words, 31% of norm- referenced schools exhibited scores on the subscale that were higher than 
the scores of the CAG schools. This was the second- highest rated of the MSQ subscales, 
indicating it as an area of strength for CAG schools. With respect to individual results. School Q 
exhibited the lowest percentile rank on the subscale, scoring lower than 91% of norm- referenced 
schools. Schools E, R and V exhibited the highest ranks, with only 4% of norm- referenced 
schools scoring higher. 
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Figure 36. Technical Resources— Percentile Ranking of Individual CAG Schools on MSCI Norm-Referenced 
Scale, and Median Score for All CAG Schools, 2010 
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Summary of the MSCI Findings 

One overarching caveat must be expressed prior to interpreting these results. These 
findings are descriptive in nature. While it is interesting and useful to compare the results of 
CAG schools with norm-referenced schools, there is not sufficient evidence to determine 
vdiether these differences between CAG schools and norm- referenced schools were 
meaningfully and statistically significant. Nor is the study design sufficient to determine if the 
survey results are directly attributable to the presence of the GAG program. With these 
limitations in mind, the following interpretations of the POSC findings are offered: 

• The MSQ identified three critical areas of need for CAG schools as a group. These 
include building capacity for peer reviewed practice, increasing expectations for student 
performance and fostering equity in practice. These were areas vdiere the median for 
CAG schools fell at or very near the 50*^ percentile. It is recommended that WVDE 
continue to focus professional development and other resources on addressing these 
deficiencies in CAG schools. 

• The MSQ identified at least three areas of strength for CAG schools as a group. These 
include the use of differentiated instruction, presence of technical resources and 
adoption of a coordinated curriculum. These are areas v\here the median for CAG 
schools fell near or above the 70*^ percentile. As such, these arieas appear to be well- 
covered in most CAG schools. However, there were some specific CAG schools vdiere the 
percentile ranks for these subscales were particularly low. WVDE researchers 
recommends continued monitoring in these schools with respect to the identified areas 
of weakness. 
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• CAG schools exhibited a small increase on five of the seven MSQ subscales during the 
course of the CAG program. The data provided in the original 2006 Edvantia report 
(Gilchrist, et al., 2006) were not sufficient to allow WVDE researchers to make any 
conclusions about the statistical significance of these changes. However, the effect size 
statistics calculated for each change indicated that the changes were negligible, at best. 
Additionally, the retrospective nature of this study does not allow WVDE researchers to 
isolate the GAG program as the agent driving this change. However, these data do 
illustrate, on a descriptive level that at least some positive change has taken place in GAG 
schools over the course of the project. 
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Discussion of Results 



The current examination yidded several results that have implications for the CAG 
program. First, with respect to summative assessment scores, there were no significant 
differences between CAG and non- CAG schools across the duration of the program. This does 
not mean that the CAG program was not successful. Achievement is only one indicator of 
program success and, as noted above, all groups under examination in CAG schools (i.e., all 
groups combined and the Black, economically disadvantaged, and students with disabilities 
subgroups), exhibited higher mean scale scores at the condusion of the program than at its 
inception in 2004. This finding hdd true for both reading/ language arts and mathematics. In 
fact, in most cases, the CAG schools exhibited a steady upward gain in average scale scores while 
the non- CAG comparison schools exhibited erratic or plateaued performance. Some of the year- 
to-year changes in CAG schools were statistically significant ( See Appendix A) . While the WVDE 
researchers cannot make the assumption that the CAG program was the sole reason for the 
gradual increase in average scale scores, it is certainly possible that the focused support 
provided by CAG haisons was a contributing factor. 

With respect to the Black student subgroup, there was an increase in mean scale scores 
at CAG schools over time, which ultimatdy resulted in Black students at the CAG schools 
exceeding the performance of Black students in non- CAG control schools for both reading/ 
anguage arts and mathematics in 2008. This finding has immense practical significance 
considering that the impetus for the CAG program rose fir)m the need to address achievement 
gaps for African American students. This finding of CAG schools outpacing comparison schools 
did not hold for all groups combined or for the economically disadvantaged and students rvith 
disabilities subgroups. The students with disabihties subgroup, in particular, exhibited quite 
erratic performance in CAG schools rA^ien compared with a gradual upward trend in non- CAG 
schools. Ultimately in 2008, the students with disabilities subgroup in CAG schools performed 
much lower than the students with disabilities subgroup in non- CAG control schools when 
examining fourth- grade scale scores. It is unclear why this is the case, but it does vrairant 
further investigation. 

In conducting post hoc analyses with all 20 CAG schools for vdiich data were available, 
WVDE researchers found that as a group, the CAG schools exhibited cumulative proficiency 
level increases that exceeded statewide increases for all groups under examination (i.e., all 
groups combined and the Black, economically disadvantaged, and students with disabilities 
subgroups). This finding held for both reading/ language arts and mathematics. However, this 
finding could potentially be an artifact of the fact that CAG schools started with larger 
achievement gaps than the state in many cases and, therefore, had more room to improve. 
Furthermore, while the CAG schools exhibited more steady improvements in proficiency rates 
over the period of the program than the state student enrollment as a whole, achievement gaps 
for the state were still narrower in 2008 than for CAG sdhools with respect to mathematics. 
Specifically, the 2008 mathematics achievement gaps between all groups combined and 
economically disadvantaged students was -7.81% ^en examining the statewide population, 
compared with -9. 14% in CAG schools; the mathematics aciiievement gap for Black students vras 
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-10.88% statewide compared with -11.19% in GAG schools. However, the gap between all groups 
combined and the students with disabilities subgroup was -33.47% in mathematics statewide, 
and only -30.51% in CAG schools. Taken together, these findings indicate that GAG schools 
were shghtly more effective than the state in closing achievement gaps in mathematics for 
students with disabilities, but not for the economically disadvantaged or Black subgroups. 

Interestingly, the results were reversed for reading/ language arts, where GAG schools 
exhibited narrower achievement gaps than the state in 2008. Specifically, the 2008 
reading/ language arts achievement gap between all groups combined and the students with 
disabilities subgroup was -38.20 statewide and -34.51% in GAG schools; the gap between all 
groups combined and the Black subgroup was -6.53% statewide and -5.35% in GAG schools. 
However, the achievement gap between all groups combined and the economically 
disadvantaged subgroup was -7.21% statewide and -7.24% in GAG schools in 2008. Taken 
together, these results illustrate that GAG schools were more effective in closing achievement 
gap in reading/ language arts for the Black and students- with-disabihties subgroups than the 
state, but neutral with respect to the economically disadvantaged subgroup. 

The questionnaires administered for this study— the Gontinuous School Improvement 
Questionnaire (GSIQ), the Perceptions of School Gulture (POSG) survey, and the Measure of 
School Gapadty for Improvement (MSQ) survey— provided some very useful data by revealing 
areas of strength and concern for many of the GAG schools. Eight areas of strength emerged for 
GAG schools across the three questionnaires: (a) ahgned and balanced curriculum; (b) positive 
learning culture; (c) use of purposeful student assessment; (d) student centered vision, 
mission, and policies; (e) teachers taking responsibility for student learning; (f) use of 
differentiated instruction; (g) presence of technical resources; and (h) use of a coordinated 
curriculum. Notably, each of the eight areas of strength is either explicitly or imphdtly ahgned 
with the GAG program standards as described in the introduction of this report. For example, 
the presence of an ahgned and balanced curriculum is dearly ahgned with Standard 3 — 
Developing Accountabihty, vdiile a positive learning culture ahgns with Standard 5— Greating 
Gommunity. While the data gathered for this study are not suffident to say that the GAG 
program caused these to become areas of strength in GAG schools, it is nonetheless an 
interesting finding how dosely they ahgn with the standards of the GAG program. Furthermore, 
these areas of strength are useful to WVDE in that they offer several potential leverage points 
that could be built upon to encourage additional successes in GAG schools. 

The three surveys also identified seven areas of concern for many of the GAG schools, and this 
information is equally useful to faculty in the GAG schools and to WVDE. These areas induded 
(a) student responsibility for learning; (b) inviting physical environment; (c) building school, 
family and community connections; (d) school-wide capadty for effective teaching; (e) peer 
reviewed practice; (f) expectations for student performance; and (g) equity in practice. The 
GAG program standards address some of these areas quite exphdtly, but others are addressed 
only imphdtly or not at ah. First, it could be said that the chief purpose of the GAG program is to 
promote equity in achievement and in practice among GAG schools. Furthermore, building 
schoolwide capadty for effective teaching is imphdtly ahgned with Standard 4— Building 
Gapadty and Standard 6— Growing Professionally. Additionahy, focusing school efforts on peer- 
reviewed practice is quite exphdtly ahgned with Standards 1, 2, 3 and 4, vdiich focus on the 
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establishment of leadership teams, professional learning communities, and fadhtating self- 
reflection among school faculty and staff. However, certain areas of weakness identified by this 
research such as building school, family, and community coimections are only partially 
addressed by the CAG standards. That is, there is a dear focus on building an internal school 
community and on coimecting schools with local colleges and universities via Standard 5 — 
Creating Community. However, the CAG standards do not explidtly address building 
coimections between schools and families or other key community partners. This is a critical 
part of the school improvement process and in creating sustainable change. Likewise, there is 
not an explidt focus within the CAG standards on building students' sense of responsibihty of 
their own learning or in rejuvenating the physical environment— two elements that are 
necessary antecedents to increased student achievement. It is important to note that no sin^e 
program can adequately address all potential areas of need in schools. However, it would be 
prudent to align future intervention in the CAG schools with the needs identified in this research 
study. 

When examining the results of the CSIQ, some interesting differences emerged among 
programmatic levds. Spedfically, CAG elementary schools, middle schools, and "other" 
configuration schools almost universally exhibited mean subscale scores that were equal to or 
exceeded the mean subscale scores for average schools from the norm- referenced sample. This 
essentially means that CAG elementary and middle schools were generally comparable to or 
higher performing than average elementary schools in terms of capacity for improvement. 
However, CAG high schools performed below the mean for average norm- referenced high 
schools on all seven subscales. This could be an artifact of the inherent differences between the 
programmatic levels or perhaps the relatively small number of CAG high schools could have 
biased the results. Regardless, this sharp contrast indicates that there are hkely several 
unmeasured differences between elementary, middle, and high schools and, as a result, the 
hmited summative assessment analyses conducted in this study do not tell the entire story of 
achievement in CAG schools. 

Focus group interviews revealed information about the context surrounding the CAG 
program and the specific work of the CAG liaisons. Specifically, the haisons reported that the 
majority of thdr technical assistance focused on addressing Standards 1, 4, and 6. Standard 
1— Bringing Focus, deals with assisting schools vdth strategic planning, developing leadership 
teams, and using data as the basis for school and classroom decision making. Standard 4— 
Building Capacity, addresses working with faculty and staff to implement standards- based 
curriculum and instruction, developing professional development that is supportive of 21st 
century teaching and learning, facilitating self-reflection, and providing support for schools' 
efforts to dose achievement gaps. Finally, Standard 6, Growing Professionally, focuses closely 
on providing support for continuous improvement through high quality and ongoing 
professional development. Within these standards, five strands or fod of services emerged. 
These induded, (a) increased student learning and achievement, (b) providing guidance and 
support to schools as opposed to exphdt direction, (c) increasing open communication with 
school staff, (d) fadlitating and following up with professional devdopment and (e) focusing 
on using data to drive instruction and assessment. 
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The information from focus group interviews revealed the challenges and barriers 
associated with the CAG program including lack of preparation time during the day, requiring 
them to prepare for their work with teachers, principals, and county administrators in the 
evenings and on weekends. Other responses to the question about challenges and barriers 
included (a) getting in the door, (b) money, (c) lack of leadership or resistance to leadership, (d) 
support, (e) retirements, and (f) maintaining a positive culture. 

Limitations 

This study was retrospective in nature. The limited availability of some data imposed 
constraints on the analyses WVDE researchers were able to conduct. Staff did not employ a 
methodology adequate to make causal inferences about the impact of the GAG program. This 
report is intended to stand among other evidence collected by the WVDE Office of Title 11, 111, 
and System Support during the course of the GAG program, and should be viewed as a single 
component rather than as the entire portfolio of evidence. 
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The WVDE Office of Research makes the following broad recommendations given the 
findings presented in this report: 

WVDE researchers recommend continued monitoring of the achievement gaps in CAG 
schools and statewide. The Office of Research is poised to accomplish this work via the annual 
updating of the Qosing the Achievement Gap report. 

Despite a lack of statistical significance for the fourth- grade assessment data examined, 
the CAG program was associated with a variety of successful program outcomes including 
marginally higher post-test perceptions of school culture, capacity for improvement, and 
continuous school improvement on most item subscales. Additionally, for the majority of cases, 
the CAG schools made steady progress in narrowing achievement gaps for all subgroups in both 
reading/ language arts and mathematics. While the research study conducted is not sufficient to 
determine M^ether the CAG program caused these changes, it is likely that the support provided 
by the CAG program was helpful for CAG schools. It is also evident that there is still room for 
improvement in these schools. The Office of Research recommends continued support in order 
to capitalize on this success. The CAG haisons have already built substantial rapport with these 
schools and would be very valuable in these future efforts. 

If the CAG program continues, ongoing evaluation should be conducted fir)m the outset 
in order to more rigorously determine the program's success in achieving its formative and 
summative goals. Because the data for this study were compiled retrospectively, it is impossible 
for WVDE researchers to determine a causal link between the CAG program and the outcomes 
reported herein. While another evaluation would likely be subject to similar deficiencies due to 
the lack of a true experiment, it would certainly allow for a more credible study of program 
impact. Additionally, ongoing evaluation is critical to guiding programmatic adjustments to the 
services dehvered via the program. 

The current study revealed a great deal of valuable information about the areas of 
strength and areas of concern in individual CAG schools. This information could be potentially 
useful in guiding future intervention efforts in the CAG schools. The Office of Research has 
provided these results to the program director and recommends that they be shared with the 
individual schools involved in the CAG program and perhaps with other offices within the 
WVDE that are working with these schools to impact school improvement. Specifically, the areas 
of strength identified by this study could be used as leverage points within these schools and 
areas of concern could be used to provide a focus for additional support services. 

The information from focus group interviews revealed the challenges and barriers 
associated with the CAG program. WVDE should learn from these challenges and design 
intervention programs that are better suited to avoid these potential pitfalls. 
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Appendix A. Null Hypothesis Significance Testing and Results of T Tests Between Program Years 



Null Hypothesis Significance Testing 

CAG Experimental Schools (n=12) versus Control Schools (n=12) 
Ho: pi = p2 Accepted otherwise Rejected 
Mathematics and Reading/Language Arts 



Hoi: There is no statistical significant difference in Mathematics Scale Scores in 2004 for 
All students between experimental schools (n=12) and control schools (n=12) where p<.05. 

Ho2: There is no statistical significant difference in Mathematics Mean Scale Scores in 2004 
for black students between experimental schools (n=10] and control schools (n=10) where p<.05. 

Ho3: There is no statistical significant difference in Mathematics Mean Scale Scores in 2004 
for students with socially economically disadvantage (Low SES) between experimental schools 
(n=12) and control schools (n=12) where p<.05. 

Ho4: There is no statistical significant difference in Mathematics Mean Scale Scores in 2004 
for students with disabilities (SWD) between experimental schools (n=12) and control schools 
(n=12) where p<.05 Ho5: There is no statistical significant difference in Mathematics Scale 
Scores in 2005 for All students between experimental schools (n=12] and control schools (n=12] 
where p<.05. 

Ho6: There is no statistical significant difference in Mathematics Mean Scale Scores in 2005 
for black students between experimental schools (n=10) and control schools (n=10) where p<.05. 

Ho7: There is no statistical significant difference in Mathematics Mean Scale Scores in 2005 
for students with Low SES between experimental schools (n=12) and control schools (n=12) 
where p<.05. 

Ho8: There is no statistical significant difference in Mathematics Mean Scale Scores in 2005 
for SWD between experimental schools (n=12] and control schools (n=12) where p<.05. 

Ho9: There is no statistical significant difference in Mathematics Scale Scores in 2006 for 
All students between experimental schools (n=12) and control schools (n=12) where p<.05. 

HolO: There is no statistical significant difference in Mathematics Mean Scale Scores in 
2006 for black students between experimental schools (n=10) and control schools (n=10) where 
p<.05. 

Holl: There is no statistical significant difference in Mathematics Mean Scale Scores in 
2006 for students with Low SES between experimental schools (n=12] and control schools (n=12] 
where p<.05. 

Hol2: There is no statistical significant difference in Mathematics Mean Scale Scores in 

2006 for SWD between experimental schools (n=12) and control schools (n=12] where p<.05. 

Hol3: There is no statistical significant difference in Mathematics Scale Scores in 2007 for 
All students between experimental schools (n=12) and control schools (n=12) where p<.05. 

Hol4: There is no statistical significant difference in Mathematics Mean Scale Scores in 

2007 for black students between experimental schools (n=10) and control schools (n=10] where 
p<.05. 
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Hol5: There is no statistical significant difference in Mathematics Mean Scale Scores in 
2007 for students with Low SES between experimental schools (n=12) and control schools (n=12) 
where p<.05. 

Hol6: There is no statistical significant difference in Mathematics Mean Scale Scores in 

2007 for SWD between experimental schools (n=12) and control schools (n=12) where p<.05. 

Hol7: There is no statistical significant difference in Mathematics Scale Scores in 2008 for 
All students between experimental schools (n=12] and control schools (n=12] where p<.05. 

Hol8: There is no statistical significant difference in Mathematics Mean Scale Scores in 

2008 for black students between experimental schools [n=10) and control schools (n=10) where 
p<.05. 

Hol9: There is no statistical significant difference in Mathematics Mean Scale Scores in 
2008 for students with Low SES between experimental schools (n=12) and control schools (n=12) 
where p<.05. 

Ho20: There is no statistical significant difference in Mathematics Mean Scale Scores in 
2008 for SWD between experimental schools (n=12) and control schools (n=12) where p<.05. 

Ho21: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2004 for All students between experimental schools (n=12] and control schools (n=12] 
where p<.05. 

Ho22: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2004 for black students between experimental schools (n=10) and control schools 
(n=10) where p<.05. 

Ho23: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2004 for or students with Low SES between experimental schools (n=12) and control 
schools (n=12) where p<.05. 

Ho24: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2004 for SWD between experimental schools (n=12) and control schools (n=12] where 
p<.05. 

Ho25: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in 2005 for All students between experimental schools (n=12] and control schools (n=12) where 
p<.05. 

Ho26: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2005 for black students between experimental schools (n=10] and control schools 
(n=10] where p<.05. 

Ho27: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2005 for or students with Low SES between experimental schools (n=12] and control 
schools (n=12) where p<.05. 

Ho28: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2005 for SWD between experimental schools (n=12) and control schools (n=12) where 
p<.05. 
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Ho29: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in 2006 for All students between experimental schools (n=12) and control schools (n=12) where 
p<.05. 

Ho30: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2006 for black students between experimental schools (n=10) and control schools 
(n=10) where p<.05. 

Ho31: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2006 for or students with Low SES between experimental schools (n=12) and control 
schools (n=12] where p<.05. 

Ho32: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2006 for SWD between experimental schools (n=12) and control schools (n=12) where 
p<.05. 

Ho33: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in 2007 for All students between experimental schools (n=12] and control schools (n=12) where 
p<.05. 

Ho34: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2007 for black students between experimental schools (n=10] and control schools 
(n=10] where p<.05. 

Ho35: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2007 for students with Low SES between experimental schools (n=12) and control 
schools (n=12) where p<.05. 

Ho36: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2007 for SWD between experimental schools (n=12) and control schools (n=12) where 
p<.05. 

Ho37: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in 2008 for All students between experimental schools (n=12] and control schools (n=12) where 
p<.05. 

Ho38: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2008 for black students between experimental schools (n=10] and control schools 
(n=10] where p<.05. 

Ho39: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2008 for or students with Low SES between experimental schools (n=12) and control 
schools (n=12) where p<.05. 

Ho40: There is no statistical significant difference in Reading/Language Arts Mean Scale 
Scores in 2008 for SWD between experimental schools (n=12) and control schools (n=12) where 
p<.05. 
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Results of r Tests between Program Years 



Table A 1. Mathematics All in Schools 2004 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


636.73 


-4.75 


12.60 


-.943 


.356 


Control 


12 


641.48 




12.10 






P<.05 Hoi: 


Null Hypotheses is accepted there is no statistical significant difference. 






Table A 2. Mathematics Black in Schools 2004 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


10 


627.94 


-3.82 


12.27 


-.637 


.532 


Control 


10 


631.76 




14.42 







p<.05 Ho2: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 3. Mathematics Low SES in Schools 2004 



Standard 

Group N Mean Difference Deviation t Significance* 



Experimental 


12 


636.40 


-4.91 


13.37 


-.914 


.370 


Control 


12 


641.32 




12.96 






P<.05 Ho3: 


Null Hypotheses is accepted there is no statistical significant difference. 






Table A 4. Mathematics SWD in Schools 2004 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


613.26 


2.73 


11.61 


.239 


.815 


Control 


12 


610.53 




37.82 






P<.05 Ho4: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 5. Mathematics All in Schools 2005 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


642.09 


-6.85 


9.94 


-1.507 


.147 


Control 


12 


648.94 




12.21 







P<.05 Ho5: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 6. Mathematics Black in Schools 2005 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


10 


634.63 


-5.69 


7.37 


-.953 


.359 


Control 


10 


640.32 




17.35 






P<.05 Ho6: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 7. Mathematics Low SES in Schools 2005 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


638.17 


-7.33 


8.96 


-1.903 


.070 


Control 


12 


645.50 




9.90 






P<.05 Ho7: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 8. Mathematics SWD in Schools 2005 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


622.29 


-3.40 


13.55 


-.512 


.614 


Control 


12 


625.69 




18.63 






P<.05 Ho8: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 9. Mathematics All in Schools 2006 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


643.53 


-3.96 


11.50 


-.842 


.409 


Control 


12 


647.49 




11.56 






P<.05 Ho9: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 10. Mathematics Black in Schools 2006 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


10 


634.25 


-6.93 


10.31 


-1.323 


.203 


Control 


10 


641.19 




12.97 






P<.05 HolO: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 11. Mathematics Low SES in Schools 2006 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


639.34 


-3.28 


10.34 


.770 


.449 


Control 


12 


642.62 




10.53 






P<.05 Holl: Null Hypotheses is accepted there is no statistical significant difference. 



Examining the Effectiveness of Closing the Achievement Gap Professional Development 87 

Demonstration Schools 



Appendix A. Null Hypothesis Significance Testing and Results of T Tests Between Program Years 



Table A 12. Mathematics SWD in Schools 2006 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


627.62 


-1.29 


14.64 


-.200 


.843 


Control 


12 


628.91 




16.86 






P<.05 Hol2: 


Null Hypotheses is accepted there is no statistical significant difference. 






Table A 13. Mathematics All in Schools 2007 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


648.92 


-.45 


7.78 


-.085 


.933 


Control 


12 


649.37 




16.53 






P<.05 Hol3: 


Null Hypotheses is accepted there is no statistical significant difference. 






Table A 14. Mathematics Black in Schools 2007 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


10 


643.78 


.572 


9.63 


.572 


.577 


Control 


10 


639.85 




19.47 






P<.05 Hol4: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 15. Mathematics Low SES in Schools 2007 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


643.86 


-2.29 


7.63 


-.453 


.657 


Control 


12 


646.15 




15.67 






P<.05 Hol5: 


Null Hypotheses is accepted there is no statistical significant difference. 






Table A 16. Mathematics SWD in Schools 2007 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


632.70 


1.95 


25.63 


.183 


.856 


Control 


12 


630.75 




26.60 






P<.05 Hol6: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 17. Mathematics All in Schools 2008 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


650.20 


-.32 


6.63 


-.082 


.935 


Control 


12 


650.52 




11.82 







P<.05 Hol7: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 18. Mathematics Black in Schools 2008 


Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


10 


649.40 4.83 


8.25 


.799 


.439 


Control 


10 


644.57 


17.26 






P<.05 Hol8: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 19. Mathematics Low in SES Schools 2008 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


639.73 -3.65 


6.32 


-1.387 


.179 


Control 


12 


643.38 


6.57 






P<.05 Hol9: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 20. Mathematics SWD in Schools 2008 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


633.13 -3.14 


17.61 


-.437 


.666 


Control 


12 


636.27 


17.59 






P<.05 Ho20: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 21. Reading/Language Arts All in Schools 2004 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


634.65 -6.29 


16.48 


-1.038 


.310 


Control 


12 


640.94 


13.00 






P<.05 Ho21: 


Null Hypotheses is accepted there is no statistical significant difference. 






Table A 22. Reading/Language Arts Black in Schools 2004 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


10 


629.88 -2.19 


5.52 


-.577 


.571 


Control 


10 


632.07 


10.66 






P<.05 Ho22: 


Null Hypotheses is accepted there is no statistical significant difference. 






Table A 23. Reading/Language Arts Low SES in Schools 2004 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


634.33 -7.07 


17.57 


-1.108 


.281 


Control 


12 


641.41 


13.43 






P<.05 Ho23: 


Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 24. Reading/Language Arts SWD in Schools 2004 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


606.97 


-.65 


13.62 


-.073 


.943 


Control 


12 


607.62 




28.12 






P<.05 Ho24: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 25. Reading/Language Arts All in Schools 2005 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


638.13 


-4.79 


8.85 


-1.115 


.278 


Control 


12 


642.92 




12.00 






P<.05 Ho25: 


Null Hypotheses is accepted there is no statistical significant difference. 






Table A 26. Reading/Language Arts Black in Schools 2005 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


10 


630.92 


1.01 


8.34 


.172 


.866 


Control 


10 


629.91 




16.44 






P<.05 Ho26: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 27. Reading/Language Arts Low SES in Schools 2005 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


632.76 


-5.77 


6.71 


-1.819 


.083 


Control 


12 


638.53 




8.71 






P<.05 Ho27: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 28. Reading/Language Arts SWD in Schools 2005 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


601.65 


-8.92 


11.54 


-.970 


.348 


Control 


12 


610.57 




29.70 






P<.05 Ho28: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 29. Reading/Language Arts All in Schools 2006 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


641.21 


-1.11 


9.48 


-.250 


.805 


Control 


12 


642.32 




12.02 






P<.05 Ho29: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 30. Reading/Language Arts Black in Schools 2006 



Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


10 


635.31 -3.27 


6.68 


-.693 


.500 


Control 


10 


638.58 


13.31 






P<.05 Ho30: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 31. Reading/Language Arts Low SES in Schools 2006 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


636.41 -1.20 


6.93 


-.305 


.764 


Control 


12 


637.61 


11.73 






P<.05 Ho31: 


Null Hypotheses is accepted there is no statistical significant difference. 






Table A 32. Reading/Language Arts SWD in Schools 2006 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


616.22 4.82 


17.59 


.570 


.575 


Control 


12 


611.40 


23.41 






P<.05 Ho32: 


Null Hypotheses is accepted there is no statistical significant difference. 






Table A 33. Reading/Language Arts All in Schools 2007 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


642.76 -2.40 


8.72 


-.537 


.598 


Control 


12 


645.16 


12.78 







P<.05 Ho33: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 34. Reading/Language Arts Black in Schools 2007 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t Significance' 


Experimental 


10 


636.13 


.62 


11.66 


.095 .921 


Control 


10 


635.51 




17.01 





P<.05 Ho34 Null Hypotheses is accepted there is no statistical significant difference. 

Table A 35. Reading/Language Arts Low SES in Schools 2007 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t Significance' 


Experimental 


12 


636.15 


-4.98 


9.56 


-1.055 .30^ 


Control 


12 


641.13 




13.27 





P<.05 Ho35: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 36. Reading/Language Arts SWD in Schools 2007 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


603.99 


-18.69 


44.73 


-1.30 


.212 


Control 


12 


622.68 




21.90 






P<.05 Ho36: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 37. Reading All Schools in 2008 












Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


644.68 


-3.32 


6.96 


-1.04 


.309 


Control 


12 


648.00 




8.56 






P<.05 Ho37: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 38. Reading Black in Schools 2008 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


10 


643.84 


5.00 


5.64 


1.02 


.328 


Control 


10 


638.84 




14.38 






P<.05 Ho38: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 39. Reading Low SES in Schools 2008 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


639.73 


-3.65 


6.32 


-1.39 


.179 


Control 


12 


643.38 




.657 






P<.05 Ho39: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 40. Reading SWD in Schools 2008 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


Experimental 


12 


617.81 


-12.02 


22.93 


-1.34 


.193 


Control 


12 


629.83 




20.86 







P<.05 Ho40: Null Hypotheses is accepted there is no statistical significant difference. 
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Null Hypothesis Significance Testing 
Title I CAG Experimental Schools 5 years 
2004 to 2008 for Mathematics and Reading Language Arts 



Ho41: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for All students from 2004 to 2005 where p<.05. 

Ho42: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for All students from 2004 to 2006 where p<.05. 

Ho43: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for All students from 2004 to 2007 where p<.05. 

Ho44: There is no statistical significant difference in Mathematics Scale Scores Scale Scores 
in the experimental group for All students from 2004 to 2008 where p<.05. 

Ho45: There is no statistical significant difference in Mathematics Scale Scores Scale Scores 
in the experimental group for All students from 2005 to 2006 where p<.05. 

Ho46: There is no statistical significant difference in Mathematics Scale Scores Scale Scores 
in the experimental group for All students from 2005 to 2007 where p<.05. 

Ho47: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for All students from 2005 to 2008 where p<.05. 

Ho48: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for All students with from 2006 to 2007 where p<.05. 

Ho49: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for All students from 2006 to 2008 where p<.05. 

Ho50: There is no statistical significant difference in Mathematics Scale Scores for Scale 
Scores in the experimental group for All students from 2007 to 2008 where p<.05. 

Ho51: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for black students from 2004 to 2005 where p<.05. 

Ho52: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for black students from 2004 to 2006 where p<.05. 

Ho53: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for black students from 2004 to 2007 where p<.05. 

Ho54: There is no statistical significant difference in Mathematics Scale Scores Scale Scores 
in the experimental group for black students from 2004 to 2008 where p<.05. 

Ho55: There is no statistical significant difference in Mathematics Scale Scores Scale Scores 
in the experimental group for black students from 2005 to 2006 where p<.05. 

Ho56: There is no statistical significant difference in Mathematics Scale Scores Scale Scores 
in the experimental group for black students from 2005 to 2007 where p<.05. 

Ho57: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for black students from 2005 to 2008 where p<.05. 
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Ho58: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for black students with from 2006 to 2007 where p<.05. 

Ho59: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for black students from 2006 to 2008 where p<.05. 

Ho60: There is no statistical significant difference in Mathematics Scale Scores for Scale 
Scores in the experimental group for black students from 2007 to 2008 where p<.05. 

Ho61: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for students with Low SES from 2004 to 2005 where p<.05. 

Ho62: There is no statistical significant difference in Mathematics Scale for Scores in the 
experimental group for students with Low SES from 2004 to 2006 where p<.05. 

Ho63: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for students with Low SES from 2004 to 2007 where p<.05. 

Ho64: There is no statistical significant difference in Mathematics Scale Scores Arts Scale 
Scores in the experimental group for students with Low SES from 2004 to 2008 where p<.05. 

Ho65: There is no statistical significant difference in Mathematics Scale Scores Scale Scores 
in the experimental group students with Low SES from 2005 to 2006 where p<.05. 

Ho66: There is no statistical significant difference in Mathematics Scale Scores Scale Scores 
in the experimental group students with Low SES from 2005 to 2007 where p<.05. 

Ho67: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group students with Low SES from 2005 to 2008 where p<.05. 

Ho68: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group students with Low SES from 2006 to 2007 where p<.05. 

Ho69: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group with students with Low SES from 2006 to 2008 where p<.05. 

Ho70: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group with students with Low SES from 2007 to 2008 where p<.05. 

Ho71: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for SWD from 2004 to 2005 where p<.05. 

Ho72: There is no statistical significant difference in Mathematics Scale for Scores in the 
experimental group for SWD from 2004 to 2006 where p<.05. 

Ho73: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for SWD from 2004 to 2007 where p<.05. 

Ho74: There is no statistical significant difference in Mathematics Scale Scores Arts Scale 
Scores in the experimental group for SWD from 2004 to 2008 where p<.05. 

Ho75: There is no statistical significant difference in Mathematics Scale Scores Scale Scores 
in the experimental group for SWD from 2005 to 2006 where p<.05. 

Ho76: There is no statistical significant difference in Mathematics Scale Scores Scale Scores 
in the experimental group for SWD from 2005 to 2007 where p<.05. 
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Ho77: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for SWD from 2005 to 2008 where p<.05. 

Ho78: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for SWD from 2006 to 2007 where p<.05. 

Ho79: There is no statistical significant difference in Mathematics Scale Scores in the 
experimental group for SWD from 2006 to 2008 where p<.05. 

Ho80: There is no statistical significant difference in Mathematics Scale Scores for Scale 
Scores in the experimental group for SWD from 2007 to 2008 where p<.05. 

Ho81: There is no statistical significant difference in Reading/Language Arts Scores in the 
experimental group for All students from 2004 to 2005 where p<.05. 

Ho82: There is no statistical significant difference in Reading/Language Arts Scale Scores in 
the experimental group for All students from 2004 to 2006 where p<.05. 

Ho83: There is no statistical significant difference in Reading/Language Arts Scale Scores in 
the experimental group for All students from 2004 to 2007 where p<.05. 

Ho84: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Scale Scores in the experimental group for All students from 2004 to 2008 where p<.05. 

Ho85: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Scale Scores in the experimental group for All students from 2005 to 2006 where p<.05. 

Ho86: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Scale Scores in the experimental group for All students from 2005 to 2007 where p<.05. 

Ho87: There is no statistical significant difference in Reading/Language Arts Scale Scores in 
the experimental group for All students from 2005 to 2008 where p<.05. 

Ho88: There is no statistical significant difference in Reading/Language Arts Scale Scores in 
the experimental group for All students with from 2006 to 2007 where p<.05. 

Ho89: There is no statistical significant difference in Reading/Language Arts Scale Scores in 
the experimental group for All students from 2006 to 2008 where p<.05. 

Ho90: There is no statistical significant difference in Reading/Language Arts Scale Scores 
for Scale Scores in the experimental group for All students from 2007 to 2008 where p<.05. 

Ho91: There is no statistical significant difference in Reading/Language Arts Scores in the 
experimental group for black students from 2004 to 2005 where p<.05. 

Ho92: There is no statistical significant difference in Reading/Language Arts Scale Scores in 
the experimental group for black students from 2004 to 2006 where p<.05. 

Ho93: There is no statistical significant difference in Reading/Language Arts Scale Scores in 
the experimental group for black students from 2004 to 2007 where p<.05. 

Ho94: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Scale Scores in the experimental group for black students from 2004 to 2008 where p<.05. 

Ho95: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Scale Scores in the experimental group for black students from 2005 to 2006 where p<.05. 
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Ho96: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Scale Scores in the experimental group for hlack students from 2005 to 2007 where p<.05. 

Ho97: There is no statistical significant difference in Reading/Language Arts Scale Scores in 
the experimental group for black students from 2005 to 2008 where p<.05. 

Ho98: There is no statistical significant difference in Reading/Language Arts Scale Scores in 
the experimental group for black students with from 2006 to 2007 where p<.05. 

Ho89: There is no statistical significant difference in Reading/Language Arts Scale Scores in 
the experimental group for black students from 2006 to 2008 where p<.05. 

HolOO: There is no statistical significant difference in Reading/Language Arts Scale Scores 
for Scale Scores in the experimental group for black students from 2007 to 2008 where p<.05. 

HolOl: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in the experimental group for students with Low SES from 2004 to 2005 where p<.05. 

Hol02: There is no statistical significant difference in Reading/Language Arts Scale for 
Scores in the experimental group for students with Low SES from 2004 to 2006 where p<.05. 

HolOS: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in the experimental group for students with Low SES from 2004 to 2007 where p<.05. 

Hol04: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Arts Scale Scores in the experimental group for students with Low SES from 2004 to 2008 where 
p<.05. 

HolOS: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Scale Scores in the experimental group for students with Low SES from 2005 to 2006 where p<.05. 

Hol06: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Scale Scores in the experimental group students with Low SES from 2005 to 2007 where p<.05. 

Ho 107: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in the experimental group for students with Low SES from 2005 to 2008 where p<.05. 

Hol08: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in the experimental group for students with Low SES from 2006 to 2007 where p<.05. 

Ho 109: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in the experimental group for students with Low SES from 2006 to 2008 where p<.05. 

HollO: There is no statistical significant difference in Reading/Language Arts Scale Scores 
for Scale Scores in the experimental group for students with Low SES from 2007 to 2008 where 
p<.05. 

Holll: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in the experimental group for SWD from 2004 to 2005 where p<.05. 

Hoi 12: There is no statistical significant difference in Reading/Language Arts Scale for 
Scores in the experimental group for SWD from 2004 to 2006 where p<.05. 

HollS: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in the experimental group for SWD from 2004 to 2007 where p<.05. 
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Hoi 14: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Arts Scale Scores in the experimental group for SWD from 2004 to 2008 where p<.05. 

HollS: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Scale Scores in the experimental group SWD from 2005 to 2006 where p<.05. 

Hoi 16: There is no statistical significant difference in Reading/Language Arts Scale Scores 
Scale Scores in the experimental group SWD from 2005 to 2007 where p<.05. 

Hoi 17: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in the experimental group SWD from 2005 to 2008 where p<.05. 

Hoi 18: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in the experimental group SWD from 2006 to 2007 where p<.05. 

Holl9: There is no statistical significant difference in Reading/Language Arts Scale Scores 
in the experimental group with Disabilities from 2006 to 2008 where p<.05. 

Hol20: There is no statistical significant difference in Reading/Language Arts Scale Scores 
for Scale Scores in the experimental group SWD from 2007 to 2008 where p<.05. 
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Results of T- Tests between Program Years 



Table A 41. Experimental Mathematics All in Schools 


Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


636.73 -5.36 


12.60 


-3.408 


.006* 


2005 


12 


642.09 


9.94 






P<.05 


Ho41: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 42. Experimental Mathematics All in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


636.73 -6.80 


12.60 


-2.822 


.017* 


2006 


12 


643.53 


11.50 







P<.05 Ho42: Null Hypotheses is rejected there is a statistical significant difference. 



Table A 43. Experimental Mathematics All in Schools 


Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


636.73 -12.19 


12.60 


-4.280 


.001* 


2007 


12 


648.92 


7.78 






P<.05 


Ho43: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 44. Experimental Mathematics All in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


636.73 -13.47 


12.60 


-4.143 


.002* 


2008 


12 


650.20 


6.64 






P<.05 


Ho44: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 45. Experimental Mathematics All in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


642.09 -1.44 


9.94 


-.612 


.553 


2006 


12 


643.53 


11.50 







P<.05 Ho45: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 46. Experimental Mathematics All in Schools 


Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


642.09 -6.83 


9.94 


-3.399 


.006* 


2007 


12 


648.92 


7.78 






P<.05 


Ho46: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 47. Experimental Mathematics All in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


642.09 -8.11 


9.94 


-3.518 


.005* 


2008 


12 


650.20 


6.64 






P<.05 


Ho47: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 48. Experimental Mathematics All in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


643.53 -5.39 


11.50 


-2.452 


.032* 


2007 


12 


648.92 


7.78 






P<.05 


Ho48 Null Hypotheses is rejected there is a statistical significant difference. 






Table A 49. Experimental Mathematics All in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


643.53 -6.67 


11.50 


-2.357 


.038* 


2008 


12 


650.20 


6.64 






P<.05 


Ho49 Null Hypotheses is rejected there is a statistical significant difference. 






Table A 50. Experimental Mathematics All in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2007 


12 


648.92 -1.28 


7.78 


-.544 


.597 


2008 


12 


650.20 


6.64 






P<.05 


Ho50: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 51. Experimental Mathematics All in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


642.94 -6.69 


12.27 


-2.291 


.048* 


2005 


10 


634.63 


7.34 






P<.05 


Ho51 Null Hypotheses is rejected there is a statistical significant difference. 
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Table A 52. Experimental Mathematics Black in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


627.94 


-6.31 


12.27 


-2.580 


.030* 


2006 


10 


634.25 




10.31 






P<.05 


Ho52: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 53. Experimental Mathematics Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


627.94 


-15.84 


12.27 


-5.051 


.001* 


2007 


10 


643.78 




9.63 







P<.05 Ho53: Null Hypotheses is rejected there is a statistical significant difference. 



Table A 54. Experimental Mathematics Black in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


627.94 


-21.46 


12.27 


-5.325 


.000* 


2008 


10 


649.40 




8.25 






P<.05 


Ho54: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 55. Experimental Mathematics Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


10 


634.63 


.38 


7.38 


.164 


.873 


2006 


10 


634.25 




10.31 






P<.05 


Ho55: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 56. Experimental Mathematics Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


10 


634.63 


-9.15 


7.78 


-6.391 


.000* 


2007 


10 


343.78 




9.63 






P<.05 


Ho56: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 57. Experimental Mathematics Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


10 


634.63 


-14.77 


7.38 


-4.515 


.001* 


2008 


10 


649.40 




8.25 






P<.05 


Ho57: Null Hypotheses is rejected there is a statistical significant difference. 
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Table A 58. Experimental Mathematics Black in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


10 


634.25 


-9.53 


10.31 


-3.851 


.004* 


2007 


10 


343.78 




9.63 






P<.05 


Ho58: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 59. Experimental Mathematics Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


10 


634.25 


-15.15 


10.31 


-5.034 


.001* 


2008 


10 


649.40 




8.25 







P<.05 Ho59: Null Hypotheses is rejected there is a statistical significant difference. 

Table A 60. Experimental Mathematics Black in Schools 



Standard 

Group N Mean Difference Deviation t Significance* 



2007 

2008 


10 

10 


643.78 

649.40 


-5.62 


9.63 

8.25 


-1.374 


.203 


P<.05 


Ho60: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 61. Experimental Mathematics SES in Schools 
















Standard 






Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2004 


12 


636.40 


-1.77 


13.37 


-.699 


.499 


2005 


12 


638.17 




8.96 






P<.05 


Ho61: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 62. Experimental Mathematics SES in Schools 
















Standard 






Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 



2004 


12 


636.40 -2.94 


13.37 


-.950 


.362 


2006 


12 


639.34 


10.34 






P<.05 


Ho62: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 63. Experimental Mathematics SES in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


636.40 -7.46 


13.37 


-2.147 


.055 


2007 


12 


643.86 


7.64 






P<.05 


Ho63: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 64. Experimental Mathematics SES in Schools 


Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


636.40 -3.33 


13.37 


-.777 


.453 


2008 


12 


639.73 


6.33 






P<.05 


Ho64: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 65. Experimental Mathematics SES in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


638.17 -1.17 


8.96 


-.518 


.615 


2006 


12 


639.34 


10.34 






P<.05 


Ho65: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 66. Experimental Mathematics SES in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


638.17 -5.69 


8.96 


-2.952 


.013* 


2007 


12 


643.86 


7.64 






P<.05 


Ho66: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 67. Experimental Mathematics SES in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


638.17 -1.56 


8.96 


-.608 


.556 


2008 


12 


639.73 


6.33 






P<.05 


Ho67: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 68. Experimental Mathematics SES in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


639.34 -4.52 


10.34 


-2.186 


.051 


2007 


12 


643.86 


7.64 






P<.05 


Ho68: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 69. Experimental Mathematics SES in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


639.34 -.39 


10.34 


-.115 


.911 


2008 


12 


639.73 


6.35 






P<.05 


Ho69: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 70. Experimental Mathematics SES in Schools 



Standard 



Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2007 


12 


643.86 


4.13 


7.64 


1.428 


.181 


2008 


12 


639.73 




6.33 






P<.05 


Ho70: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 71. Experimental Mathematics SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


613.26 


-9.03 


11.61 


-2.651 


.023* 


2005 


12 


622.29 




13.55 







P<.05 Ho71: Null Hypotheses is rejected there is a statistical significant difference. 

Table A 72. Experimental Mathematics SWD in Schools 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


613.26 


-14.36 


11.61 


-2.901 


.014* 


2006 


12 


627.62 




14.64 






P<.05 


Ho72: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 73. Experimental Mathematics SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


613.26 


-19.44 


11.61 


-2.550 


.027* 


2007 


12 


632.70 




25.63 







P<.05 Ho73: Null Hypotheses is rejected there is a statistical significant difference. 



Table A 74. Experimental Mathematics SWD in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


613.26 


-19.87 


11.61 


-3.152 


.009* 


2008 


12 


633.13 




17.61 






P<.05 


Ho74: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 75. Experimental Mathematics SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


622.29 


-5.33 


13.55 


-1.152 


.274 


2006 


12 


627.62 




14.64 






P<.05 


Ho75: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 76. Experimental Mathematics SWD in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


622.29 


-10.41 


13.55 


-1.849 


.091 


2007 


12 


632.70 




25.63 






P<.05 


Ho76: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 77. Experimental Mathematics SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


622.29 


-10.84 


12.21 


-.714 


.490 


2008 


12 


633.13 




11.82 






P<.05 


Ho77: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 78. Experimental Mathematics SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


627.62 


-5.09 


627.62 


-.952 


.361 


2007 


12 


632.70 




632.70 






P<.05 


Ho78: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 79. Experimental Mathematics SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


627.62 


-5.51 


14.64 


-.946 


.365 


2008 


12 


633.13 




17.61 






P<.05 


Ho79: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 80. Experimental Mathematics SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


632.70 


-.43 


25.63 


-.085 


.934 


2008 


12 


633.13 




17.61 






P<.05 


Ho80: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 81. Experimental Reading/Language Arts All in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


634.65 


-3.48 


16.48 


-1.130 


.282 


2005 


12 


638.13 




8.85 






P<.05 


Ho81: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 82. Experimental Reading/Language Arts All in Schools 



Standard 



Group 


N 


Mean 


Difference 


Deviation 


T 


Significance* 


2004 


12 


634.65 


-3.47 


16.48 


-1.626 


.132 


2006 


12 


641.21 




9.48 






P<.05 


Ho82: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 83. Experimental Reading/Language Arts All in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


634.65 


-8.11 


16.48 


-1.758 


.106 


2007 


12 


642.76 




8.72 







P<.05 Ho83: Null Hypotheses is accepted there is no statistical significant difference. 



Table A 84. Experimental Reading/Language Arts All in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


634.65 


-10.03 


16.48 


-2.368 


.037* 


2008 


12 


644.68 




6.96 






P<.05 Ho84: Null Hypotheses is rejected there is a statistical significant difference. 

Table A 85. Experimental Reading/Language Arts All in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


638.13 


-3.08 


8.85 


-1.143 


.185 


2006 


12 


641.21 




9.48 






P<.05 Ho85: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 86. Experimental Reading/Language Arts All in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


638.13 


-4.63 


8.85 


-1.742 


.109 


2007 


12 


642.76 




8.72 






P<.05 Ho86: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 87. Experimental Reading/Language Arts All in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


638.13 


-6.55 


8.85 


-3.044 


.011* 


2008 


12 


644.68 




6.96 






P<.05 


Ho87: Null Hypotheses is rejected there is a statistical significant difference. 
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Table A 88. Experimental Reading/Language Arts All in Schools 



Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


641.21 -1.55 


9.45 


-.583 


.572 


2007 


12 


642.76 


8.72 






P<.05 


Ho88: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 89. Experimental Reading/Language Arts All in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


641.21 -3.47 


9.48 


-1.758 


.107 


2008 


12 


644.68 


6.96 






P<.05 


Ho89: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 90. Experimental Reading/Language Arts All in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2007 


12 


642.76 -1.92 


8.72 


-.651 


.528 


2008 


12 


644.68 


6.96 






P<.05 


Ho90: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 91. Experimental Reading/Language Arts Blacks in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


629.88 -1.04 


5.52 


-.379 


.714 


2005 


10 


630.92 


8.34 






P<.05 


Ho91: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 92. Experimental Reading/Language Arts Blacks in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


629.88 -5.43 


5.52 


-2.027 


.073 


2006 


10 


635.31 


6.68 






P<.05 


Ho92: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 93. Experimental Reading/Language Arts Blacks in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


629.88 -6.25 


5.52 


-1.648 


.134 


2007 


10 


636.13 


11.66 







P<.05 Ho93: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 94. Experimental Reading/Language Arts Blacks in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


629.88 


-13.95 


5.52 


-6.881 


.000* 


2008 


10 


643.84 




5.64 






P<.05 


Ho94: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 95. Experimental Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


10 


630.92 


-4.39 


8.34 


-1.144 


.282 


2006 


10 


635.31 




6.68 






P<.05 


Ho95: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 96. Experimental Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


10 


630.92 


-5.21 


8.34 


-1.702 


.123 


2007 


10 


636.13 




11.66 






P<.05 


Ho96: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 97. Experimental Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


10 


630.92 


-12.92 


8.34 


-3.839 


.004* 


2008 


10 


643.84 




5.64 






P<.05 


Ho97: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 98. Experimental Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


10 


635.31 


-.82 


6.68 


-.188 


.855 


2007 


10 


636.13 




11.66 






P<.05 


Ho98: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 99. Experimental Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


10 


635.31 


-8.52 


6.68 


-4.047 


.003* 


2008 


10 


643.83 




5.64 






P<.05 


Ho99: Null Hypotheses is rejected there is a statistical significant difference. 
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Table A 100. Experimental Read Reading/Language Arts Black in Schools 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2007 


10 


636.13 


-7.71 


11.66 


-1.585 


.147 


2008 


10 


643.84 




5.64 






P<.05 


HolOO: Null Hypotheses is accepted there is no statistical significant difference. 




Table A 101. Experimental Reading/Language Arts Low SES in Schools 














Standard 






Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2004 


12 


634.34 


1.58 


17.57 


.332 


.746 


2005 


12 


632.76 




6.71 






P<.05 


HolOl: Null Hypotheses is accepted there is no statistical significant difference. 




Table A 102. Experimental Reading/Language Arts Low SES in Schools 














Standard 






Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2004 


12 


634.34 


-2.07 


17.57 


-.437 


.671 


2006 


12 


636.41 




6.93 







P<.05 Hol02: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 103. Experimental Reading/Language Arts Low SES in Schools 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t Significance' 


2004 


12 


634.34 


-1.81 


17.57 


-.324 .75: 


2007 


12 


636.15 




9.56 





P<.05 Hol03: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 104. Experimental Reading/Language Arts Low SES in Schools 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


634.34 


-5.39 


17.57 


-1.021 


.329 


2008 


12 


639.73 




6.33 






P<.05 


Hol04: Null Hypotheses is accepted there is no statistical significant difference. 




Table A 105. Experimental Reading/Language Arts Low SES in Schools 














Standard 






Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2005 


12 


632.76 


-3.65 


6.71 


-1.740 


.110 


2006 


12 


636.41 




6.93 






P<.05 


Hol05: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 106. Experimental Reading/Language Arts Low SES in Schools 


Group 


N 


Mean 


Standard 

Difference Deviation 


t 


Significance* 


2005 


12 


632.76 


-3.39 6.71 


-1.628 


.132 


2007 


12 


636.15 


9.56 






P<.05 


Hol06: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 107. Experimental Reading/Language Arts Low SES in Schools 






Group 


N 


Mean 


Standard 

Difference Deviation 


t 


Significance* 


2005 


12 


632.76 


-6.97 6.71 


-3.063 


.011* 


2008 


12 


639.73 


6.33 






P<.05 


Hol07: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 108. Experimental Reading/Language Arts Low SES in Schools 






Group 


N 


Mean 


Standard 

Difference Deviation 


t 


Significance* 


2006 


12 


636.41 


.26 6.93 


.098 


.924 


2007 


12 


636.15 


9.56 






P<.05 


Hol08: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 109. Experimental Reading/Language Arts Low SES in Schools 






Group 


N 


Mean 


Standard 

Difference Deviation 


t 


Significance* 


2006 


12 


636.41 


-3.32 6.93 


-1.47 


.108 


2008 


12 


639.73 


6.33 






P<.05 


Hol09: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 110. Experimental Reading/Language Arts Low SES in Schools 






Group 


N 


Mean 


Standard 

Difference Deviation 


t 


Significance* 


2007 


12 


636.15 


-3.58 9.56 


-1.054 


.314 


2008 


12 


639.73 


6.33 






P<.05 


HollO: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 111. Experimental Reading/Language Arts SWD in Schools 






Group 


N 


Mean 


Standard 

Difference Deviation 


t 


Significance* 


2004 


12 


606.97 


5.32 13.62 


1.132 


.282 


2005 


12 


601.65 


11.54 






P<.05 


Holll: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 112. Experimental Reading/Language Arts SWD in Schools 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t Significance* 


2004 


12 


606.97 


-9.26 


13.62 


-1.754 .107 


2006 


12 


616.23 




17.59 





P<.05 Holl2: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 113. Experimental Reading/Language Arts SWD in Schools 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


606.97 


2.98 


13.62 


.259 


.801 


2007 


12 


603.99 




44.73 






P<.05 


Holl3: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 114. Experimental Reading/Language Arts SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


606.97 


-10.84 


13.62 


-1.782 


.102 


2008 


12 


617.81 




22.93 






P<.05 


Holl4: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 115. Experimental Reading/Language Arts SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


601.65 


-14.57 


11.54 


-2.131 


.056 


2006 


12 


616.22 




17.59 






P<.05 


Holl5: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 116. Experimental Reading/Language Arts SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


601.65 


-2.34 


11.54 


-.184 


.857 


2007 


12 


603.99 




44.73 






P<.05 


Holl6: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 117. Experimental Reading/Language Arts SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


601.65 


-16.16 


11.54 


-2.088 


.061 


2008 


12 


617.81 




22.93 






P<.05 


Holl7: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 118. Experimental Reading/Language Arts SWD in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


616.22 


12.23 


17.59 


1.104 


.293 


2007 


12 


603.99 




44.74 






P<.05 


H0II8: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 119. Experimental Reading/Language Arts SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


616.22 


-1.59 


17.59 


-.271 


.792 


2008 


12 


617.81 




22.93 






P<.05 


Holl9: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 120. Experimental Reading/Language Arts SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2007 


12 


603.99 


-13.82 


44.74 


-1.454 


.174 


2008 


12 


617.81 




22.93 







P<.05 H0I2O: Null Hypotheses is accepted there is no statistical significant difference. 
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Null Hypothesis Significance Testing 

CAG Control Group Schools 

2004 to 2008 for Mathematics and Reading Language Arts 

HolSl. There is no statistical significant difference in Mathematics Scale Scores in the control group for 
All students from 2004 to 2005 where p<.05. 

Hol32: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
All students from 2004 to 2006 where p<.05. 

Hol33: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
All students from 2004 to 2007 where p<.05. 

Hol34: There is no statistical significant difference in Mathematics Scale Scores Scale Scores in the control 
group for All students from 2004 to 2008 where p<.05. 

Hol35: There is no statistical significant difference in Mathematics Scale Scores Scale Scores in the control 
group for All students from 2005 to 2006 where p<.05. 

Hol36: There is no statistical significant difference in Mathematics Scale Scores Scale Scores in the control 
group for All students from 2005 to 2007 where p<.05. 

Hol37: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
All students from 2005 to 2008 where p<.05. 

Hol38: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
All students with from 2006 to 2007 where p<.05. 

Hol39: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
All students from 2006 to 2008 where p<.05. 

Hol40: There is no statistical significant difference in Mathematics Scale Scores for Scale Scores in the 
control group for All students from 2007 to 2008 where p<.05. 

Hol41: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
black students from 2004 to 2005 where p<.05. 

Hol42: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
black students from 2004 to 2006 where p<.05. 

Hol43: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
black students from 2004 to 2007 where p<.05. 

Hol44: There is no statistical significant difference in Mathematics Scale Scores Scale Scores in the control 
group for black students from 2004 to 2008 where p<.05. 

Hol45: There is no statistical significant difference in Mathematics Scale Scores Scale Scores in the control 
group for black students from 2005 to 2006 where p<.05. 

Hol46: There is no statistical significant difference in Mathematics Scale Scores Scale Scores in the control 
group for black students from 2005 to 2007 where p<.05. 

Hol47: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
black students from 2005 to 2008 where p<.05. 

Hol48: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
black students with from 2006 to 2007 where p<.05. 
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Hol49: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
black students from 2006 to 2008 where p<.05. 

HolSO: There is no statistical significant difference in Mathematics Scale Scores for Scale Scores in the 
control group for black students from 2007 to 2008 where p<.05. 

HolSl: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
students with Low SES from 2004 to 2005 where p<.05. 

Hol52: There is no statistical significant difference in Mathematics Scale for Scores in the control group 
for students with Low SES from 2004 to 2006 where p<.05. 

Hol53: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
students with Low SES from 2004 to 2007 where p<.05. 

Hol54: There is no statistical significant difference in Mathematics Scale Scores Arts Scale Scores in the 
control group for students with Low SES from 2004 to 2008 where p<.05. 

Hol55: There is no statistical significant difference in Mathematics Scale Scores Scale Scores in the control 
group for students with Low SES from 2005 to 2006 where p<.05. 

Hol56: There is no statistical significant difference in Mathematics Scale Scores Scale Scores in the control 
group for students with Low SES from 2005 to 2007 where p<.05. 

Hol57: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
students with Low SES from 2005 to 2008 where p<.05. 

Hol58: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
students with Low SES from 2006 to 2007 where p<.05. 

Hol59: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
students with Low SES from 2006 to 2008 where p<.05. 

Hol60: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
students with Low SES from 2007 to 2008 where p<.05. 

Hol61: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
SWD from 2004 to 2005 where p<.05. 

Hol62: There is no statistical significant difference in Mathematics Scale for Scores in the control group 
for SWD from 2004 to 2006 where p<.05. 

Hol63: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
SWD from 2004 to 2007 where p<.05. 

Hol64: There is no statistical significant difference in Mathematics Scale Scores Arts Scale Scores in the 
control group for SWD from 2004 to 2008 where p<.05. 

Hol65: There is no statistical significant difference in Mathematics Scale Scores Scale Scores in the control 
group for SWD from 2005 to 2006 where p<.05. 

Hol66: There is no statistical significant difference in Mathematics Scale Scores Scale Scores in the control 
group for SWD from 2005 to 2007 where p<.05. 

Hol67: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
SWD from 2005 to 2008 where p<.05. 

Hol68: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
SWD from 2006 to 2007 where p<.05. 
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Hol69: There is no statistical significant difference in Mathematics Scale Scores in the control group for 
SWD from 2006 to 2008 where p<.05. 

Hol70: There is no statistical significant difference in Mathematics Scale Scores for Scale Scores in the 
control group for SWD from 2007 to 2008 where p<.05. 

Hol71: There is no statistical significant difference in Reading/Language Arts Scores in the control group 
for All students from 2004 to 2005 where p<.05. 

Hol72: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for All students from 2004 to 2006 where p<.05. 

Hol73: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for All students from 2004 to 2007 where p<.05. 

Hol74: There is no statistical significant difference in Reading/Language Arts Scale Scores Scale Scores in 
the control group for All students from 2004 to 2008 where p<.05. 

Hol75: There is no statistical significant difference in Reading/Language Arts Scale Scores Scale Scores in 
the control group for All students from 2005 to 2006 where p<.05. 

Hol76: There is no statistical significant difference in Reading/Language Arts Scale Scores Scale Scores in 
the control group for All students from 2005 to 2007 where p<.05. 

Hol77: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for All students from 2005 to 2008 where p<.05. 

Hol78: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for All students with from 2006 to 2007 where p<.05. 

Hol79: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for All students from 2006 to 2008 where p<.05. 

Hol80: There is no statistical significant difference in Reading/Language Arts Scale Scores for Scale Scores 
in the control group for All students from 2007 to 2008 where p<.05. 

Hol81: There is no statistical significant difference in Reading/Language Arts Scores in the control group 
for black students from 2004 to 2005 where p<.05. 

Hol82: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for black students from 2004 to 2006 where p<.05. 

Hol83: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for black students from 2004 to 2007 where p<.05. 

Hol84: There is no statistical significant difference in Reading/Language Arts Scale Scores Scale Scores in 
the control group for black students from 2004 to 2008 where p<.05. 

Hol85: There is no statistical significant difference in Reading/Language Arts Scale Scores Scale Scores in 
the control group for black students from 2005 to 2006 where p<.05. 

Hol86: There is no statistical significant difference in Reading/Language Arts Scale Scores Scale Scores in 
the control group for black students from 2005 to 2007 where p<.05. 

Hol87: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for black students from 2005 to 2008 where p<.05. 

Hol88: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for black students with from 2006 to 2007 where p<.05. 
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Hol89: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for black students from 2006 to 2008 where p<.05. 

Hol90: There is no statistical significant difference in Reading/Language Arts Scale Scores for Scale Scores 
in the control group for black students from 2007 to 2008 where p<.05. 

Hol91: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for students with Low SES from 2004 to 2005 where p<.05. 

Hol92: There is no statistical significant difference in Reading/Language Arts Scale for Scores in the 
control group for students with Low SES from 2004 to 2006 where p<.05. 

Hol93: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for students with Low SES from 2004 to 2007 where p<.05. 

Hol94: There is no statistical significant difference in Reading/Language Arts Scale Scores Arts Scale 
Scores in the control group for students with Low SES from 2004 to 2008 where p<.05. 

Hol95: There is no statistical significant difference in Reading/Language Arts Scale Scores Scale Scores in 
the control group for students with Low SES from 2005 to 2006 where p<.05. 

Hol96: There is no statistical significant difference in Reading/Language Arts Scale Scores Scale Scores in 
the control group for students with Low SES from 2005 to 2007 where p<.05. 

Hol97: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for students with Low SES from 2005 to 2008 where p<.05. 

Hol98: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for students with Low SES from 2006 to 2007 where p<.05. 

Hol99: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for students with Low SES from 2006 to 2008 where p<.05. 

Hol90: There is no statistical significant difference in Reading/Language Arts Scale Scores for Scale Scores 
in the control group for students with Low SES from 2007 to 2008 where p<.05. 

Hol91: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for SWD from 2004 to 2005 where p<.05. 

Hol92: There is no statistical significant difference in Reading/Language Arts Scale for Scores in the 
control group for SWD from 2004 to 2006 where p<.05. 

Hol93: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for SWD from 2004 to 2007 where p<.05. 

Hol94: There is no statistical significant difference in Reading/Language Arts Scale Scores Arts Scale 
Scores in the control group for SWD from 2004 to 2008 where p<.05. 

Hol95: There is no statistical significant difference in Reading/Language Arts Scale Scores Scale Scores in 
the control group for SWD from 2005 to 2006 where p<.05. 

Hol96: There is no statistical significant difference in Reading/Language Arts Scale Scores Scale Scores in 
the control group for SWD from 2005 to 2007 where p<.05. 

Hol97: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for SWD from 2005 to 2008 where p<.05. 

Hol98: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for SWD from 2006 to 2007 where p<.05. 
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Hol99: There is no statistical significant difference in Reading/Language Arts Scale Scores in the control 
group for SWD from 2006 to 2008 where p<.05. 

Ho200: There is no statistical significant difference in Reading/Language Arts Scale Scores for Scale Scores 
in the control group for SWD from 2007 to 2008 where p<.05. 
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Results of T- Tests between Program Years 






Table A 121. Control Mathematics All in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


641.48 


-7.46 


12.10 


-2.910 


.014* 


2005 


12 


648.94 




12.21 






P<.05 


Hol21: Null Hypotheses rejected there is a statistical significant difference. 






Table A 122. Control Mathematics All in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


641.48 


-6.01 


12.10 


-2.996 


.012* 


2006 


12 


647.49 




11.56 







P<.05 Hol22: Null Hypotheses is rejected there is a statistical significant difference. 

Table A 123. Control Mathematics All in Schools 



Standard 

Group N Mean Difference Deviation t Significance* 



2004 

2007 


12 

12 


641.48 

649.37 


-7.89 


12.10 

16.53 


-2.107 


.059 


P<.05 


Hol23: Null Hypotheses is accepted there is no statistical significant difference. 




Table A 124. Control Mathematics All in Schools 


















Standard 






Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2004 


12 


641.48 


-9.04 


12.10 


-4.457 


.001* 


2008 


12 


650.52 




11.82 






P<.05 


Hol24: Null Hypotheses rejected there is a statistical significant difference. 






Table A 125. Control 1 Mathematics All in Schools 


















Standard 






Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2005 


12 


648.94 


1.45 


12.21 


.935 


.370 


2006 


12 


647.49 




11.56 







P<.05 Hol25: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 126. Control Mathematics All in Schools 


Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 648.94 


-.43 


12.21 


-.132 


.898 


2007 


12 649.37 




16.53 






P<.05 


Hol26: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 127. Control Mathematics All in Schools 










Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 648.94 


-1.58 


12.21 


-.714 


.490 


2008 


12 650.52 




11.82 






P<.05 


Hol27: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 128. Control Mathematics All in Schools 










Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 647.49 


-1.88 


11.56 


-.680 


.510 


2007 


12 649.37 




16.53 






P<.05 


Hol28: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 129. Control Mathematics All in Schools 










Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 647.49 


-3.03 


11.56 


-1.925 


.080 


2008 


12 650.52 




11.82 






P<.05 


Hol29: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 130. Control Mathematics All in Schools 










Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2007 


12 649.37 


-1.15 


16.53 


-.360 


.726 


2008 


12 650.52 




11.82 






P<.05 


Hol30: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 131. Control Mathematics Black in Schools 










Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 631.76 


-8.56 


14.42 


-1.919 


.087 


2005 


10 640.32 




17.35 






P<.05 


Hol31: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 132. Control Mathematics Black in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t Significance* 


2004 


10 


631.76 


-9.43 


14.42 


-5.172 .001* 


2006 


10 


641.19 




12.97 





P<.05 Hol32: Null Hypotheses is rejected there is a statistical significant difference. 

Table A 133. Control Mathematics Black in Schools 



Standard 

Group N Mean Difference Deviation t Significance* 



2004 

2007 


10 

10 


631.76 

639.85 


-8.09 


14.42 

19.47 


-1.478 


.173 


P<.05 


Hol33: Null Hypotheses is accepted there is no statistical significant difference. 




Table A 134. Control Mathematics Black in Schools 
















Standard 






Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2004 


10 


637.76 


-12.81 


14.42 


-3.051 


.014* 


2008 


10 


344.57 




17.26 






P<.05 


Hol34: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 135. Control Mathematics Black in Schools 
















Standard 






Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2005 


10 


640.32 


-.87 


17.35 


-.218 


.832 


2006 


10 


641.19 




12.97 







P<.05 Hol35: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 136. Control Mathematics Black in Schools 



Group 

2005 

2007 


N 

10 

10 


Mean 

640.32 

639.85 


Difference 

.47 


Standard 

Deviation 

17.35 

19.47 


t 

.109 


Significance* 

.915 


P<.05 


Hol36: Null Hypotheses is accepted there is no statistical significant difference. 




Table A 137. Control Mathematics Black in Schools 
















Standard 






Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2005 


10 


640.32 


-4.25 


17.35 


-.968 


.358 


2008 


10 


644.57 




17.26 






P<.05 


Hol37: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 138. Control Mathematics Black in Schools 



Standard 



Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2006 


10 


641.19 


1.34 


12.97 


.245 


.812 


2007 


10 


639.85 




19.47 






P<.05 


Hol38: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 139. Control Mathematics Black in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


10 


641.19 


-3.38 


12.97 


-.774 


.459 


2008 


10 


644.57 




17.26 






P<.05 


Hol39: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 140. Control Mathematics Black in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2007 


10 


639.85 


-4.72 


19.46 


-.835 


.425 


2008 


10 


644.57 




17.26 






P<.05 


Hol40: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 141. Control Mathematics Low SES in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


641.32 


-4.18 


12.96 


-1.370 


.198 


2005 


12 


645.50 




9.90 






P<.05 


Hol41: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 142. Control Mathematics Low SES in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


641.32 


-1.30 


12.96 


-.485 


.638 


2006 


12 


642.62 




10.53 






P<.05 


Hol42: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 143. Control Mathematics Low SES in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


641.32 


-4.82 


12.96 


-1.229 


.245 


2007 


12 


646.15 




15.67 







P<.05 Hol43: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 144. Control Mathematics Low SES in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


641.32 


-2.06 


12.96 


-.626 


.544 


2008 


12 


643.38 




6.58 






P<.05 


Hol44: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 145. Control Mathematics Low SES in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


645.50 


2.88 


9.90 


2.180 


.052 


2006 


12 


642.62 




10.53 






P<.05 


Hol45: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 146. Control Mathematics Low SES in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


645.50 


-.65 


9.90 


-.210 


.837 


2007 


12 


646.15 




15.67 






P<.05 


Hol46: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 147. Control Mathematics Low SES in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


645.50 


2.12 


9.90 


.751 


.468 


2008 


12 


643.38 




6.57 






P<.05 


Hol47: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 148. Control Mathematics Low SES in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


642.62 


-3.53 


10.53 


-1.456 


.173 


2007 


12 


646.15 




15.67 






P<.05 


Hol48: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 149. Control Mathematics Low SES in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


642.62 


-.76 


10.53 


-.284 


.782 


2008 


12 


643.38 




6.58 







P<.05 Hol49: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 150. Control Mathematics Low SES in Schools 



Group 


N 


Mean 


Difference 


Standard 

Deviation 


t Significance* 


2007 


12 


646.15 


2.77 


15.67 


.761 .463 


2008 


12 


643.38 




6.58 





P<.05 Hol50: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 151. Control Mathematics SOW in Schools 



Standard 

Group N Mean Difference Deviation t Significance* 



2004 


12 


610.53 


-15.16 


37.82 


-1.871 


.088 


2005 


12 


625.69 




18.62 






P<.05 


Hol51: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 152. Control Mathematics SWD in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


610.53 


-18.38 


37.82 


-2.075 


.062 


2006 


12 


629.91 




16.86 







P<.05 Hol52: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 153. Control Mathematics SWD in Schools 



Standard 



Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2004 


12 


610.53 


-20.23 


37.82 


-2.860 


.016* 


2007 


12 


630.75 




26.60 






P<.05 


Hol53: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 154. Control Mathematics SWD in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


610.53 


-25.74 


37.82 


-3.54 


.005* 


2008 


12 


636.27 




17.59 






P<.05 


Hol54: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 155. Control Mathematics SWD in Schools 










Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


625.69 


-3.22 


18.86 


-.780 


.452 


2006 


12 


628.91 




16.86 






P<.05 


Hol55: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 156. Control Mathematics SWD in Schools 


Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 625.69 


-5.06 


18.62 


-.679 


.511 


2007 


12 630.75 




26.60 






P<.05 


Hol56: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 157. Control Mathematics SWD in Schools 










Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 625.69 


-10.57 


18.63 


-2.358 


.038* 


2008 


12 636.27 




17.59 






P<.05 


Hol57: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 158. Control Mathematics SWD in Schools 










Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 628.91 


-1.84 


16.86 


-.302 


.768 


2007 


12 630.75 




26.60 






P<.05 


Hol58: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 159. Control Mathematics SWD in Schools 










Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 628.91 


-7.36 


16.86 


-1.563 


.146 


2008 


12 636.27 




17.59 






P<.05 


Hol59: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 160. Control Mathematics SWD in Schools 










Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2007 


12 630.75 


-5.52 


26.60 


-1.094 


.297 


2008 


12 636.27 




17.59 






P<.05 


Hol60: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 161. Control Reading/Language Arts All in Schools 








Group 


N Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 640.94 


-1.98 


13.00 


-.632 


.540 


2005 


12 642.92 




12.00 






P<.05 


Hol61: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 162. Control Reading/Language Arts All in Schools 



Standard 



Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2004 


12 


640.94 


-1.37 


13.00 


-.502 


.625 


2006 


12 


642.31 




12.02 






P<.05 


Hol62: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 163. Control Reading/Language Arts All in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


640.94 


-4.22 


13.00 


-.984 


.346 


2007 


12 


645.16 




12.78 







P<.05 Hol63: Null Hypotheses is accepted there is no statistical significant difference. 



Table A 164. Control Reading/Language Arts All in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


640.94 


-7.06 


13.00 


-2.361 


.038* 


2008 


12 


648.00 




8.56 






P<.05 Hol64: Null Hypotheses is rejected there is a statistical significant difference. 

Table A 165. Control Reading/Language Arts All in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


642.92 


.60 


12.00 


.401 


.696 


2006 


12 


642.32 




12.02 






P<.05 Hol65: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 166. Control Reading/Language Arts All in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


642.92 


-2.24 


12.00 


-.860 


.406 


2007 


12 


645.16 




12.78 






P<.05 Hol66: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 167. Control Reading/Language Arts All in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


642.92 


-5.08 


12.02 


-2.806 


.017* 


2008 


12 


648.00 




12.78 






P<.05 


Hol67: Null Hypotheses is rejected there is a statistical significant difference. 
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Table A 168. Control Reading/Language Arts All in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


642.32 


-2.84 


12.02 


-1.180 


.253 


2007 


12 


645.16 




12.78 






P<.05 


Hol68: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 169. Control Reading/Language Arts All in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


642.32 


-5.68 


12.02 


-2.382 


.036* 


2008 


12 


648.00 




8.56 






P<.05 


Hol69: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 170. Control Reading/Language Arts All in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2007 


12 


645.16 


-2.84 


12.78 


-1.077 


.305 


2008 


12 


648.00 




8.56 






P<.05 


Hol70: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 171. Control Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


632.07 


2.16 


10.66 


.422 


.683 


2005 


10 


629.91 




16.45 






P<.05 


Hol71: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 172. Control Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


632.07 


-6.51 


10.66 


-2.704 


.024* 


2006 


10 


638.58 




13.31 






P<.05 


Hol72: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 173. Control Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


632.07 


-3.44 


10.66 


-.563 


.587 


2007 


10 


635.51 




17.02 






P<.05 


Hol73. Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 174. Control Reading/Language Arts Black in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


10 


632.07 


-6.77 


10.66 


-1.952 


.083 


2008 


10 


638.84 




14.38 






P<.05 


Hol74: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 175. Control Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


10 


629.91 


-8.67 


16.45 


-1.594 


.145 


2006 


10 


638.58 




13.31 






P<.05 


Hol75: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 176. Control Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


10 


629.91 


-5.60 


16.45 


-2.288 


.048* 


2007 


10 


635.51 




17.02 






P<.05 


Hol76: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 177. Control Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


10 


629.91 


-8.93 


16.45 


-1.631 


.137 


2008 


10 


638.84 




14.38 






P<.05 


Hol77: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 178. Control Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


10 


638.58 


3.06 


13.31 


.498 


.631 


2007 


10 


635.51 




17.02 






P<.05 


Hol78: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 179. Control Reading/Language Arts Black in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


10 


638.58 


-.26 


13.31 


-.072 


.944 


2008 


10 


638.84 




14.38 






P<.05 


Hol79: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 180. Control Reading/Language Arts Black in Schools 



Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2007 


10 


635.51 -3.33 


17.02 


-.568 


.572 


2008 


10 


638.84 


14.38 






P<.05 


Hol80: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 181. Control Reading/Language Arts Low SES in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


641.41 2.88 


13.43 


.706 


.495 


2005 


12 


638.53 


8.71 






P<.05 


Hol81: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 182. Control Reading/Language Arts Low SES in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


641.41 3.80 


13.43 


1.137 


.280 


2006 


12 


637.61 


11.73 






P<.05 


Hol82: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 183. Control Reading/Language Arts Low SES in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


641.41 .28 


13.43 


.054 


.958 


2007 


12 


641.13 


13.27 






P<.05 


Hol83: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 184. Control Reading/Language Arts Low SES in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


641.41 -1.97 


13.43 


-.558 


.588 


2008 


12 


643.38 


6.58 






P<.05 


Hol84: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 185. Control Reading/Language Arts Low SES in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


638.53 .92 


8.71 


.478 


.642 


2006 


12 


637.61 


11.73 







P<.05 Hol85: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 186. Control Reading/Language Arts Low SES in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


638.53 


-2.60 


8.71 


-1.064 


.310 


2007 


12 


641.13 




13.27 






P<.05 


Hol86: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 187. Control Reading/Language Arts Low SES in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


638.53 


-4.85 


8.71 


-2.445 


.033* 


2008 


12 


643.38 




6.56 






P<.05 


Hol87: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 188. Control Reading/Language Arts Low SES in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


637.61 


-3.52 


11.73 


-1.215 


.250 


2007 


12 


641.13 




13.27 






P<.05 


Hol88: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 189. Control Reading/Language Arts Low SES in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


637.61 


-5.77 


11.73 


-2.202 


.050* 


2008 


12 


643.38 




6.58 






P<.05 


Hol89: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 190. Control Reading/Language Arts Low SES in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2007 


12 


641.13 


-2.25 


13.27 


-.741 


.474 


2008 


12 


643.38 




6.58 






P<.05 


Hol90: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 191. Control Reading/Language Arts SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


607.62 


-2.95 


28.12 


-.330 


.748 


2005 


12 


610.57 




29.70 






P<.05 


Hol91: Null Hypotheses is accepted there is no statistical significant difference. 
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Table A 192. Control Reading/Language Arts SWD in Schools 



Standard 



Group 


N 


Mean 


Difference 


Deviation 


t 


Significance* 


2004 


12 


607.62 


-3.78 


28.12 


-.707 


.494 


2006 


12 


611.40 




23.41 






P<.05 


Hol92: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 193. Control Reading/Language Arts SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2004 


12 


607.62 


-15.06 


28.12 


-2.252 


.046* 


2007 


12 


622.68 




21.90 







P<.05 Hol93: Null Hypotheses is rejected there is a statistical significant difference. 

Table A 194. Control Reading/Language Arts SWD in Schools 



Standard 



Group 


N 


Mean Difference 


Deviation 


t 


Significance* 


2004 


12 


607.62 -22.21 


28.12 


-2.891 


.015* 


2008 


12 


629.83 


20.86 






P<.05 


Hol94: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 195. Control Reading/Language Arts SWD in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


610.57 -.83 


29.70 


-.103 


.919 


2006 


12 


611.40 


23.41 







P<.05 Hol95: Null Hypotheses is accepted there is no statistical significant difference. 

Table A 196. Control Reading/Language Arts SWD in Schools 



Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


610.57 -12.11 


29.70 


-1.660 


.125 


2007 


12 


622.68 


21.90 






P<.05 


Hol96: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 197. Control Reading/Language Arts SWD in Schools 








Group 


N 


Mean Difference 


Standard 

Deviation 


t 


Significance* 


2005 


12 


610.57 -19.26 


29.70 


-3.036 


.011* 


2008 


12 


629.83 


20.86 






P<.05 


Hol97: Null Hypotheses is rejected there is a statistical significant difference. 
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Table A 198. Control Reading/Language Arts SWD in Schools 


Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


611.40 


-11.28 


23.41 


-1.852 


.091 


2007 


12 


622.68 




21.90 






P<.05 


Hol98: Null Hypotheses is accepted there is no statistical significant difference. 






Table A 199. Control Reading/Language Arts SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2006 


12 


611.40 


-18.43 


23.41 


-2.660 


.022* 


2008 


12 


629.83 




20.86 






P<.05 


Hol99: Null Hypotheses is rejected there is a statistical significant difference. 






Table A 200. Control Reading/Language Arts SWD in Schools 








Group 


N 


Mean 


Difference 


Standard 

Deviation 


t 


Significance* 


2007 


12 


622.68 


-7.15 


21.90 


-1.312 


.216 


2008 


12 


629.83 




20.86 







P<.05 Ho200: Null Hypotheses is accepted there is no statistical significant difference. 
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Appendix B. Focus Group Protocol for CAG Liaisons 



CAG Focus Group Protocol 

November 3, 2009 

1. What are your primary roles and responsibilities as a CAG? Could you please give me some examples 
of the services you are typically asked to provide? 



2. In your view, what would you state as the primary goals and objectives of the CAG program? How 
have you worked with your schools to help them meet these goals? Examples ? (Optional probe: How 
successful would you say the CAG program has been on a scale of 1 - 10 with 1 being "not at all 
successful" and 10 being "very successful.") 

3. How many schools do you currently serve? Within those schools, what types of professional 
development or training/resources have you provide? 



4. How long have you been working with each of your schools, and how often would you say you get to 
visit each of them? 

• How many hours per visit? 

• How many visits per month/semester/year? 

• Could you tell me about what happens during a typical site visit? 

5. What are some changes you have seen in the schools with which you work? What type(s) of evidence 
of these changes have you seen? (Optional probe: what feedback have you received from the schools 
that you work with?) 

6. What have been some of the greatest successes and challenges you have encountered in working 
with your schools? 

7. What training and/or preparation did you receive to prepare you for this work? What has been 
most/least useful? 

8. Is there anything else you would like to say about your work as a CAG that we didn't talk about 
today? 
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